​​Movement Practices Lab 1: Pose Detection with PoseNet
​​In this activity, we will turn your computer into a software development studio. There’s a lot to install, and a lot of steps to turn your general-purpose computer into a software development computer, but we are getting a big chunk of it out the way all at once, so that we can then turn our focus to learning and building.

​
​Technologies

​
​PoseNet
​​PoseNet is a machine learning vision model that estimates the pose of a person in an image or video by estimating where key body joints are.
​​
​​PoseNet runs in Python, JavaScript, and C++. In class, we will be using the JavaScript version.
​​
​
​p5.js
​​p5.js is a JavaScript library with the goal of making coding accessible to artists, designers, educators, and beginners.
​​
​​p5.js is inspired by—and very similar to—Processing. If you know Processing, many of the concepts in p5.js will look familiar (and vice versa). Unlike Processing, p5.js uses JavaScript, and runs in a web browser (including on mobile devices). 
​​
​
​ml5.js
​​ml5.js simplifies the creation and exploration of programs that use artificial intelligence (machine learning) algorithm in a web page.
​​
​​In this assignment, we will employ ml5.js as a means to access PoseNet.
​​
​​ml5.js is developed and maintained by NYU ITP. 

​
​Install PoseNet on Your Computer

​​In this activity, you will install a simple program that uses PoseNet, and run it on your laptop.
​​
​​This will also get you minimal familiarity with: the git version control program; the Visual Studio Code program editor; how to use npm (the Node Package Manager) to install JavaScript programs.
​​
​​Follow these steps. Don’t worry if you don’t understand everything: (1) you don’t have to, and (2) the teaching team will circulate to help you through them.

​
​A. Setup for Software Development

​​In which you create accounts and install tools that allow you to do collaborative software development. This gets you set for the course; also, for many other IMA courses. This needs to be done once each time you buy a new computer (download Visual Studio Code and Cmder), or re-boot your online identity (creating a GitHub account).
​​
  1. ​​Create an account on GitHub.com.
  1. ​​Download and install Visual Studio Code. [If you already have a favorite IDE / program editor, such as Atom, you can stick with that instead.]
  1. ​​Open Visual Studio Code. Turn on File > Autosave. (This directs Visual Studio Code to save the changes to file you’re editing, whenever you click on file pane, window, or program.) 
  1. ​​Windows: Download and install Cmder. Use the “Download Full” button to install the git-for-windows version.
  1. ​​Install Node.js:
  • ​​macOS: (1) install HomeBrew. (2) Open the Terminal application. (3) Enter brew install node​ into the Terminal window.

​
​B. Clone the Project Code

​​In which you make your own copy of the PoseNet application. You will do something similar two or three more times in this course, as we distribute additional workshop applications or you start your own projects.
​​
  1. ​​Visit https://github.com/osteele/p5pose. At the top of the page, click the Fork button.
  1. ​​This will create a new repository (GitHub project), and take you to a page for that project. For example, if your GitHub login is “nyush-student”, you will be directed to https://github.com/nyush-student/p5pose​.
  1. ​​Click the “Clone or download” button. Click the Copy (clipboard) icon next the URL. (The URL will be https://github.com/nyush-student/p5pose.git​, where nyush-student​ will be replaced by your GitHub login.)
  1. ​​[Macintosh] Open the Terminal application, if it is not already open. [Windows] Launch the Cmder application that you downloaded under Setup. Henceforth, both the macOS Terminal and the Window Cmder will be referred to as the “terminal”.
  1. ​​In a terminal, enter the text git clone https://github.com/nyush-student/p5pose.git​, where https://github.com/nyush-student/p5pose.git​ is the URL you pasted above. (On macOS, the computer may inform you that developer tools are not installed, and ask you whether you want to install Xcode. Click“Install”.)
​​
​​The result should be a folder named p5pose​ in your home directory, that contains the project files. You can examine this folder in the macOS Finder or Windows Explorer. You can also open the folder in Visual Studio Code.
​​

​
​C. Install the Libraries

​​In which you download the JavaScript packages (libraries) that are used to build and run the starter application. You will need to do this each time you download and run a JavaScript project on your computer.
​​
  1. ​​In the terminal, enter cd p5pose​. This “aims” that terminal window (changes its working directory) to the project folder.
  1. ​​Enter npm install​.

​
​D. Run the Code

​​In which you run PoseNet on your laptop and webcam.
​​
​​In the terminal, enter npm start​.
​​
​​This should open the application in a browser window. The browser may ask permission to use your laptop’s webcam. Grant it.
​​
​​Now you’ve PoseNet running on your laptop. Run it, and explore how well it works with wherever your laptop camera is aimed.

​
​E. Make a Change

​​In which you verify that you can change the behavior of the PoseNet starter kit program.
​​
  1. ​​Use Visual Studio Code to open the p5pose​ folder.
  1. ​​In the Explorer pane on the left, click on src​ to disclose the files inside the p5pose/src​ folder.
  1. ​​Click on sketch.js​ to open the file in an editor pane.
  1. ​​Find the line ellipse(keypoint.position.x, keypoint.position.y, 10, 10);​. Change one of the 10​ s to a 20​.
  1. ​​Save your change (cmd-S).
  1. ​​Now look back at the browser window. It should have reloaded, with slightly different appearance.

​
​F. Publish Your Change

​​In which you publish a change to GitHub. This tests the mechanics of submitting your assignment.
​​
​​We will be using git and GitHub for assignments. You will commit your work to the git repository that you created above, and then push your local changes to the public repository in your account on GitHub.com.
​​There are two ways to do this:
  1. ​​Using Visual Studio Code:
  1. ​​Click the “Source Control” icon (a circle at the bottom that branches to two circles above) in the left toolbar.
  1. ​​Enter text into the Message text box. This is a human-readable description of your changes; for example, “Modified the keypoint display”.
  1. ​​Click the check box at the top of the pane.
  1. ​​Answer “Yes” to the “There are no staged changes to commit” dialog box.
  1. ​​At the bottom left of the window, to the right of the word “master”, there is a circle followed by the text “0↓1↑”. Click this text.
  1. ​​Using the command line. In the terminal, change directories into the project directory and enter the following commands:
​​git add .
​​git commit -m "Modified the keypoint display"
​​git push
​​