3D Scene Composition
A scene is composed of two layers:
- Background 2D image
- Foreground 3D rendered object or scene
Background 2D images are rendered directly to offscreen canvas while foreground 3D content is first rendered on a webgl canvas then transferred to the offscreen 2D canvas.
2019.12.20 - Commit d5261ba - MAJOR: 3D rendering using three.js (Part 2 of n)
MAJOR: 3D rendering using three.js (Part 2 of n) · issalab/mkturk@d5261ba
Building from rendering prototype code, implemented 3d rendering using threejs into mkturk task flow. Created mkturk_trialqueue_scene.js for 3D images to mimic functionality of mkturk_trialqueue fo...
MAJOR: 3D rendering using three.js (Part 2 of n) · issalab/mkturk@d5261ba • github.com
2020.01.10 - Commit a5d8e78 - MAJOR: 3D rendering using three.js (Part 3 of 4)
MAJOR: 3D rendering using three.js (Part 3 of 4) · issalab/mkturk@a5d8e78
Handles 3D rendering at different 2D screen positions by moving the viewport before each render. This gives a veridical "token" with a head on camera as opposed to perspective view replac...
MAJOR: 3D rendering using three.js (Part 3 of 4) · issalab/mkturk@a5d8e78 • github.com
2020.02.27 - Commit 04f93cf - MAJOR: 3D rendering using three.js (Part 4 of 5) -- Saving images
2020.04.26 - Commit d840961 - MAJOR: THREEJS Animation
And then there were 3d animations...Nice work !
MAJOR: THREEJS Animation · issalab/mkturk@d840961
Code for 3D animation in Sample taskscreen Expands on current mkturk task screen sequences by including frame-by-frame updates within a task screen. This is done by taking rendering params from sc...
MAJOR: THREEJS Animation · issalab/mkturk@d840961 • github.com
2020.05.30 - Commit 0090c02 - MAJOR: Streamline mkturk 3D scene & 2D image display
MAJOR: Streamline mkturk 3D scene & 2D image display · issalab/mkturk@0090c02
Under-the-hood changes in how mkturk displays movies to be more compatible with universal frame-by-frame sequence display. Merges Sample & Test behavior so that both can be movies (only Sample ...
MAJOR: Streamline mkturk 3D scene & 2D image display · issalab/mkturk@0090c02 • github.com
2020.06.02 - Commit 832cc4f - MAJOR: THREEJS Animation - Morph between meshes (part 3 of 4)
MAJOR: THREEJS Animation - Morph between meshes (part 3 of 4) · issalab/mkturk@832cc4f
Mesh names are provided as keypoints in Scene params file under object's morphTarget field. All morphs are full strength in that morph is done fully to next keypoint (eg, object 1 fully becomes...
MAJOR: THREEJS Animation - Morph between meshes (part 3 of 4) · issalab/mkturk@832cc4f • github.com
2020.06.06 - Commit 38b9531 - MAJOR: THREEJS Animation - Morph between meshes (part 4 of 4)
MAJOR: THREEJS Animation - Morph between meshes (part 4 of 4) · issalab/mkturk@38b9531
Update for controlling timeing of mesh morphs: For example, morphTarget[i] = ["neptune","neptune"] means that original object will reach neptune at durationMS/2 and stay neptune...
MAJOR: THREEJS Animation - Morph between meshes (part 4 of 4) · issalab/mkturk@38b9531 • github.com
Last but not least in 3d animation, morphing meshes. This could be used for manipulating expression or identity of a face. Here's an RSVP movie showing an elias -> neptune identity morph on top of the other animations in lighting, position, rotation, size.
2D Image Filtering
As a post-processing step, can be independently applied to 2D image & 3D scene layers. Included filters: brightness, contrast, blur, hue rotate, reverse contrast, grayscale, sepia tone.
2020.09.26 - Commit - MAJOR: Merge pull request from issalab/yj_twig
Add 2D object & image augmentation using css canvas filters
Image augmentations are here! Courtesy of , now you can apply various 2D filters immediately before image display. 3D foreground objects and 2D background images can be filtered separately. Filters parameters can be animated just like any other scene param by entering an array of values. (edited)
- Timing: window.requestAnimationFrame displays canvas content locked to the next screen refresh thanks in part to improved timing precision from the performance.now() function
- Resolution: retina displays and in particular those on tablets and smartphones are pushing dpi to new levels from >200 on tablets up to 500 dpi on the newest google pixel phone. This allows greater detail in image content. Alternatively, this allows viewers to be positioned much closer to screen, reducing space requirements, while still delivering rich content.
Double Buffered Rendering
An invisible, offscreen canvas is used for fast double buffered rendering of 2D & 3D content while the current frame is displayed on the visible, onscreen canvas. When rendering is complete, the fully rendered offscreen canvas’s content is simply committed to the visible onscreen canvas as a no copy bitmap saving memory.
Consistent Image Display Across Devices
To ensure that an image is displayed at the same size in inches across devices, MkTurk detects the device based on a hardware fingerprint and looks up it’s display pixel density per inches to convert from screen pixels to physical inches (*requires device database).
A single json file contains the user-controlled task settings (~50 parameters). Here the user specifies which scenes to render and whether the subject should do a fixation, stimulus-response, match-to-sample, or same-different task.