Movement Practices Lab 1: Pose Detection with PoseNet
In this activity, we will turn your computer into a software development studio. There’s a lot to install, and a lot of steps to turn your general-purpose computer into a software development computer, but we are getting a big chunk of it out the way all at once, so that we can then turn our focus to learning and building.
Technologies
PoseNet
PoseNet is a machine learning vision model that estimates the pose of a person in an image or video by estimating where key body joints are.
PoseNet runs in Python, JavaScript, and C++. In class, we will be using the JavaScript version.
p5.js
p5.js is a JavaScript library with the goal of making coding accessible to artists, designers, educators, and beginners.
p5.js is inspired by—and very similar to—Processing. If you know Processing, many of the concepts in p5.js will look familiar(and vice versa). Unlike Processing, p5.js uses JavaScript, and runs in a web browser(including on mobile devices).
ml5.js
ml5.js simplifies the creation and exploration of programs that use artificial intelligence(machine learning) algorithm in a web page.
In this assignment, we will employ ml5.js as a means to access PoseNet.
ml5.js is developed and maintained by NYU ITP.
Install PoseNet on Your Computer
In this activity, you will install a simple program that uses PoseNet, and run it on your laptop.
This will also get you minimal familiarity with: the git version control program; the Visual Studio Code program editor; how to use npm(the Node Package Manager) to install JavaScript programs.
Follow these steps. Don’t worry if you don’t understand everything:(1) you don’t have to, and(2) the teaching team will circulate to help you through them.
A. Setup for Software Development
In which you create accounts and install tools that allow you to do collaborative software development. This gets you set for the course; also, for many other IMA courses. This needs to be done once each time you buy a new computer(download Visual Studio Code and Cmder), or re-boot your online identity(creating a GitHub account).
Download and install Visual Studio Code.[If you already have a favorite IDE / program editor, such as Atom, you can stick with that instead.]
Open Visual Studio Code. Turn on File > Autosave.(This directs Visual Studio Code to save the changes to file you’re editing, whenever you click on file pane, window, or program.)
Windows: Download and install Cmder. Use the“Download Full” button to install the git-for-windows version.
In which you make your own copy of the PoseNet application. You will do something similar two or three more times in this course, as we distribute additional workshop applications or you start your own projects.
This will create a new repository(GitHub project), and take you to a page for that project. For example, if your GitHub login is“nyush-student”, you will be directed to https://github.com/nyush-student/p5pose.
Click the“Clone or download” button. Click the Copy(clipboard) icon next the URL.(The URL will be https://github.com/nyush-student/p5pose.git, where nyush-student will be replaced by your GitHub login.)
[Macintosh] Open the Terminal application, if it is not already open.[Windows] Launch the Cmder application that you downloaded under Setup. Henceforth, both the macOS Terminal and the Window Cmder will be referred to as the“terminal”.
In a terminal, enter the text git clone https://github.com/nyush-student/p5pose.git, where https://github.com/nyush-student/p5pose.git is the URL you pasted above.(On macOS, the computer may inform you that developer tools are not installed, and ask you whether you want to install Xcode. Click“Install”.)
The result should be a folder named p5pose in your home directory, that contains the project files. You can examine this folder in the macOS Finder or Windows Explorer. You can also open the folder in Visual Studio Code.
Technologies
Install PoseNet on Your Computer
A. Setup for Software Development
B. Clone the Project Code