Learning with Interactive 3D Models

3D Virtual-Rocks for Improving Rock-Category Learning in Physical Geology
Scott Brande, UAB, Birmingham, AL; sbrande@uab.edu; IUSE #1937389; UAB IRB-300004223Robert Nosofsky, Indiana Univ., Bloomington, IN; nosofsky@indiana.edu; IUSE #1937361

NSF Collaborative Research - Award

Of course! A match made in cyberspace. First met virtually on suggestion of a common colleague. Nosofsky's expertise in cognitive science and the theory of category learning provides match to test Brande's technology in the messy real-world environment of students in labs.

Why Is This Research Important?

Every human develops abstract categories based on experience with concrete objects that stand as examples with diagnostic properties of the category: birds, pots and pans, friends, even minerals and rocks.

What We Don't Know - Yet!

Can students learn to identify a granite, for example, from studying multiple examples of interactive 3D models of physical samples? We don't know - yet! Engagement conditions designed by Robert Nosofsky through collaborative award and programmed by Brande-directed software engineers in Rockviewer.

Training #1 - Traditional for Generations

Learning to identify rock type as normally practiced in an introductory physical geology lab. For training under Condition #1, student engages only rocks-in-the-box for the same amount of time as the other conditions.

Training #2 - 3D Interactive Models Only

For training under Condition #2, student engages only 3D interactive models for the same amount of time as the other conditions.

Training #3 - Hybrid

For training under Condition #3, student engages physical rocks-in-the-box for 1/2 the time, and 3D interactive models for 1/2 the time.

Methods & Procedures

There are benefits of training with high-quality interactive virtual rocks over learning from traditional kit of physical samples. Important to understand that "feeling" a rock sample does not convey any information (haptic) that needed for a diagnostic identification of the rock name. "Rough" vs "Smooth" does not lead to a particular rock type identification.

Balanced Statistical Design

All students experience all three training conditions, once each for igneous, sedimentary, and metamorphic rock identification. Each rock type uses one lab period. Experiment runs across three labs.

1st Cohort - No Stats Yet.

Second cohort just completed spring semester. Data to compile and analyze during summer 2022.

Into the Tech Weeds - Start with a Physical Rock

Rock samples about 1-3 inches in size. Rocks for 3D models totaled 4 samples of each of 10 rock types - igneous, sedimentary, metamorphic. 40 igneous rocks, 40 sedimentary rocks, 36 metamorphic rocks (we couldn't acquire 4 samples of one metamorphic rock type).

Home Studio - 3 LED Panel Lights, Canon RP, Macro Lens...
(20 min / rock * 116 rocks)

NSF funds for equipment to produce high quality images needed for high quality models. Rock is positioned, and a focus-stacked shoot of 10 images is made. Rock is rotated 10 to 15 degrees, another shoot. Rock is rotated through 360 degrees (manually on a turntable -not shown). Rock is flipped upside down and rotated to shoot bottom. Rock is rotated 90 degrees to shoot the sides (if needed).

Input: 10 Images Each 10 Degrees of Rotation on Turntable. 600 to 1500 files / Rock ( 7 to 10 hours / Rock * 116 rocks)

Focus-stacking is needed for detail of small 1 to 2 inch rock samples. Maintaining focus at nearly 1-to1 magnification across a rock sample is difficult. Camera body has built-in software to shoot 10 multiple images at each focal distance of rock from front to back. Focus stacking processing software (Canon) composites into a single image only the in-focus portion of each of the 10 images in the stack.

Our Custom Code for Masking Background (Fast! 2 minutes / rock)

Our custom code is needed to eliminate background before model making.

Metashape Software to Build Model (1/2 to 1 hour/rock)

Commercial software needed for processing composited images (60 to 100+) into 3D model. Processing time can take up to 1 hour per model.

116 Models for Student Training

40 rock samples of 10 types of igneous rock. 40 rock samples of of 10 types of sedimentary rock. 36 rock samples of 10 types of metamorphic rock.

3D Model for Interactive Engagement (Public)

Example 3D model (not part of research set). Public model served on Sketchfab platform.

Rockviewer Platform for Student Training

Funding used for software engineers to use gaming engine (Unity) and Microsoft database to administer functions for student training sequences.

Video of Student Training and Testing

2 minute video showing deployment in the classroom.

Poster at NSF-IUSE Summit 2022

With links.