AR Local Multiplayer: Technical and UX Concept
Local multiplayer should be the shining point of AR games where players actually feels doing real stuffs together. 

With the new power of ARKit and ARCore, the phones are now be able to interpret locations even without image processing. This leads to the possibility of allowing players to interact with each other in AR world.

  • For a simple example: We all have this fantasy of shooting each other in real world like in FPS games. Now we may actually can with just normal phone.

Inspiration was successfully crowdfunded in April 2016 and recently announced to have went into mass production. 
The project is a bit ambiguous, combining two separated game loops: a geolocation loop where player claims territory, and an AR first-person-shooter gameplay where player uses a laser-tag like accessory.
It is branded as a Massive Multiplayer Laser Tag game includes a phone app and an accessory to be attached to the phone. This is the trailer for the product.
The downside of this product is the player is required to have an accessory to enjoy the game. The upside is, it is supposed to be very accurate, smooth and easy to set up.

  • The research being proposed here aims to solve the problem without external accessories and allows anyone with a smartphone to enjoy AR local multiplayer.

Technical Concept

The main problem of AR local multiplayer is for a phone to know where other phones location is in 3D world. My proposition here is setting up a transformation matrix between 2 phones and calculate the relative position based on absolute position recognized on each phone.

Mathematical Model

  • Let there are 2 phones named A and B, with independent 3D worlds named WAW_A and WBW_B.
  • At setup step, with AR camera ray cast:
  • On phone A, find the positions PA/AP_{A/A} of phone A and PB/AP_{B/A} of phone B in WAW_A, 
  • On phone B, find the positions PA/BP_{A/B} of phone A and PB/BP_{B/B} of phone B in WBW_B, 
  • While the zero point in the two worlds are different, the vector from phone A to phone B in the two worlds should fit a constant transformation. Therefore we can form the transform matrix TBAT_{B\rightarrow A} to project a vector from WBW_B to WAW_A as follow:

  • PB/APA/A=TBA (PB/BPA/B)P_{B/A} - P_{A/A} = T_{B\rightarrow A}\ (P_{B/B} - P_{A/B}) 
  • \Rightarrow TBA=(PB/APA/A) (PB/BPA/B)1T_{B\rightarrow A} = (P_{B/A} - P_{A/A})\ (P_{B/B} - P_{A/B}) ^ {-1}.

  • Let the position in WBW_B of the zero point POA/AP_{OA/A} of WAW_A to be POA/BP_{OA/B}, we have:

  • PB/APOA/A=TBA (PB/BPOA/B)P_{B/A} - P_{OA/A} = T_{B\rightarrow A}\ (P_{B/B} - P_{OA/B})
  • PB/APOA/A=TBAPB/BTBAPOA/B\Rightarrow P_{B/A} - P_{OA/A} = T_{B\rightarrow A} P_{B/B} - T_{B\rightarrow A} P_{OA/B}
  • TBAPOA/B=PB/A+POA/A+TBAPB/B\Rightarrow T_{B\rightarrow A} P_{OA/B} = - P_{B/A} + P_{OA/A} + T_{B\rightarrow A} P_{B/B}
  • POA/B=TBA1(PB/A+POA/A+TBAPB/B)\Rightarrow P_{OA/B} = T_{B\rightarrow A} ^ {-1} (- P_{B/A} + P_{OA/A} + T_{B\rightarrow A} P_{B/B})

  • Edit: Since POA/A=(0,0,0)P_{OA/A} = (0, 0, 0), we can remove it.
  • POA/B=TBA1(PB/A+TBAPB/B)P_{OA/B} = T_{B\rightarrow A} ^ {-1} (- P_{B/A} + T_{B\rightarrow A} P_{B/B})

  • At anytime:
  • On phone B, find the locations: PB/BP_{B/B} of phone B in WBW_B, 
  • On phone A, to find PB/AP_{B/A} we have:

  • PB/APOA/A=TBA (PB/BPOA/B)P_{B/A} - P_{OA/A} = T_{B\rightarrow A}\ (P_{B/B} - P_{OA/B})
  • PB/A=TBA (PB/BPOA/B)POA/A\Rightarrow P_{B/A} = T_{B\rightarrow A}\ (P_{B/B} - P_{OA/B}) - P_{OA/A}.

  • Edit: Since POA/A=(0,0,0)P_{OA/A} = (0, 0, 0), we can remove it.
  • PB/A=TBA (PB/BPOA/B)P_{B/A} = T_{B\rightarrow A}\ (P_{B/B} - P_{OA/B}).

  • In short, these are the formulas: