MkTurk Guide

3D Scene Composition

A scene is composed of two layers:
  • Background 2D image
  • Foreground 3D rendered object or scene
Background 2D images are rendered directly to offscreen canvas while foreground 3D content is first rendered on a webgl canvas then transferred to the offscreen 2D canvas.




2019.12.20 - Commit d5261ba - MAJOR: 3D rendering using three.js (Part 2 of n)
2020.01.10 - Commit a5d8e78 - MAJOR: 3D rendering using three.js (Part 3 of 4)
2020.02.27 - Commit 04f93cf - MAJOR: 3D rendering using three.js (Part 4 of 5) -- Saving images

2020.04.26 - Commit d840961 - MAJOR: THREEJS Animation
And then there were 3d animations...Nice work @You-Nah!

2020.05.30 - Commit 0090c02 - MAJOR: Streamline mkturk 3D scene & 2D image display
2020.06.02 - Commit 832cc4f - MAJOR: THREEJS Animation - Morph between meshes (part 3 of 4)
2020.06.06 - Commit 38b9531 - MAJOR: THREEJS Animation - Morph between meshes (part 4 of 4)
Last but not least in 3d animation, morphing meshes. This could be used for manipulating expression or identity of a face. Here's an RSVP movie showing an elias -> neptune identity morph on top of the other animations in lighting, position, rotation, size.

2D Image Filtering

As a post-processing step, image filtering can be independently applied to 2D image & 3D scene layers. Included filters: brightness, contrast, blur, hue rotate, reverse contrast, grayscale, sepia tone.

2020.09.26 - Commit f60dd95- MAJOR: Merge pull request #17 from issalab/yj_twig
Add 2D object & image augmentation using css canvas filters
Image augmentations are here! Courtesy of @You-Nah, now you can apply various 2D filters immediately before image display. 3D foreground objects and 2D background images can be filtered separately. Filters parameters can be animated just like any other scene param by entering an array of values. (edited) 

Visual Display

Advances in display hardware (retina display) and software (javascript canvas) improved the speed and precision of image display in the browser. Notably:
  • Timing: window.requestAnimationFrame displays canvas content locked to the next screen refresh thanks in part to improved timing precision from the performance.now() function
  • Resolution: retina displays and in particular those on tablets and smartphones are pushing dpi to new levels from >200 on tablets up to 500 dpi on the newest google pixel phone. This allows greater detail in image content. Alternatively, this allows viewers to be positioned much closer to screen, reducing space requirements, while still delivering rich content.

Double Buffered Rendering

An invisible, offscreen canvas is used for fast double buffered rendering of 2D & 3D content while the current frame is displayed on the visible, onscreen canvas. When rendering is complete, the fully rendered offscreen canvas’s content is simply committed to the visible onscreen canvas as a no copy bitmap saving memory.

Consistent Image Display Across Devices

To ensure that an image is displayed at the same size in inches across devices, MkTurk detects the device based on a hardware fingerprint and looks up it’s display pixel density per inches to convert from screen pixels to physical inches (*requires device database).

Task Specification

A single json file contains the user-controlled task settings (~50 parameters). Here the user specifies which scenes to render and whether the subject should do a fixation, stimulus-response, match-to-sample, or same-different task.

Scene Specification