​​SpeedUpAmerica - Moving Forward
  • ​​This document is outdated as of the June meeting we decided to not build a separate backend service. All the GIS work will be in the v2 frontend+backend service. Otherwise the work is pretty much the same, but no need for a consolidation phase and we will stick with the current MySQL DB.
​​
​​The document outlines long-term high-level development strategy for SpeedUpAmerica. This document breaks future work into three phases:
​​
  1. ​​Phase 1.5 - Vector Tilesets
  1. ​​Phase 2 - Frontend v2
  1. ​​Phase 2.5 - New Boundaries
  1. ​​Phase 3 - Data Import Consolidation
​​
  • ​​Items struckout are done.
​​
​​In the current architecture, DNS records for speedupamerica.com point to an Application Load Balancer that distributes traffic across ECS Service Tasks running the Ruby on Rails service contained in the SpeedUpAmerica repo. An additional repo for handling DB scheme migrations has been created. This allows us to modify our DB independently of the main service; there are numerous availability benefits to this approach. The database has several tables holding boundaries, speed test submissions, and ISP data.
​​

​
​Phase 1.5 - Vector Tilesets

​​We need to implement vector tilesets if we want add more states, like Washington and Idaho. This work can be done in parallel with Phase 2 and 2.5. The bulk of the work is getting the boundaries, use Tippecanoe to generate the vector tilesets, upload them to Mapbox, update our map implementation to MapBox GL JS, implement an endpoint that returns data to join to the tilesets, adding logic to show our tilsets, and join+style the vectors based on our data.

​
​Phase 2 - Frontend v2

​​One of the primary complaints about the current service is that it’s in Ruby and uses a framework that most people are not familiar with and is in decline. We also have a new UI design that would require a complete rework of our HTML, CSS, and JS. This a good time to move serving of the Frontend to a new service using modern and commonly used framework. 
​​
​​In this phase we create a new repo/service that handles serving traditional pages implementing the redesign. It will communicate with the same DB as the v1 (Ruby) application. Data import logic will remain in the v1 codebase until Phase 3.
​​
​​Once this phase is the Ruby application and it’s ECS Service are shutdown and removed. The migration and the data import tasks are left intact.

​
​Phase 2.5 - New Boundaries

​​Ruby on Rails does not handle the MySQL GIS data types or functions well. To support arbitrary boundaries and real-time point-in-polygon lookups we need to load the geometries into the database. To do this we need to move away from Rails and implement a database for storing boundaries and lookups. This phase can be done in parallel with Phase 1.5 and 2.
​​
​​As part of this phase we will setup a PostgreSQL w/ PostGIS, or MySQL DB, and a GIS service that will form the foundation for a new Submissions service. The current Submissions will use the new service to get a list of geometries during speed test submission, resulting in real-time point-in-polygon lookups and results immediately showing up on the map. By the end of this phase we will have switched to vector tilesets and no longer have to store boundaries in the current MySQL database. At that time we will begin supporting additional boundaries as they’ll just be additional results in the point-in-polygon lookups, vector tilesets, and columns in the submissions table.

​
​Phase 3 - Data Import Consolidation

​​The data import logic written is Ruby is still in place that needs to be rewritten. Either the logic is added to the v2 repo or a new repo is created the rewrite go there. Much of the logic will be the same, but with an eye for scaling the process to a national level.

​
​Conclusion

​​Getting to the point of handling national level traffic, arbitrary boundaries, and our new UI will require multiple phases. If we were to try to do this from scratch we would not be able to gain market traction while developing the system. This approach is also more agile, we can respond to issues sooner and make adjustments as we learn from operating the existing solution. It also helps train people to develop in business/product environment where developers and designers can’t “stop the world” while we correct mistakes/issues.
​​