Data Collection

We started off with testing the Beacon Scanner app on one Android phone available amongst the team, using Post Test Server V2 (https://ptsv2.com/) to log the collected data. PTS-V2 is a service for developers testing clients that POST and GET things over HTTP, so they know what is being transmitted. While PTS-V2 allows for free data-logging onto an online server, it can only handle a few requests (usually only 1 to 2 minutes of data requests) before crashing or self-deleting. As a result, we were limited in collecting data for a maximum of 2 minutes at any given time, post which we had to export the logs manually to our system and start the process all over again. 

​

Because large scale data collection was not feasible with the above methodology, we were given access to an exclusive data-collection server for logging detected beacon signals. To test this method, we successfully carried out 12 minutes of data collection in Downtown Brooklyn on Aug 6th, 2021. 

​

To test the method out on a bigger scale and understand complexities of data collection, we carried out a pilot data collection exercise in New York City. We used the existing Beacon Scanner app and walked around three neighborhoods in NYC with retail stores concentration to scan beacon signals: SoHo, Times Square and Downtown Brooklyn, and were able to build a dataset of 6513 beacon signal records. The geographic locations of scanning sites and times of scans were manually logged on an excel spreadsheet, along with stores in the vicinity. 

​

To automate the process of GPS collection, we decided to use the GPS logger app which collects and logs GPS location of the user, along with timestamps. In order to build a dataset where detected beacon signals and GPS locations are detected at the same time, our methodology was to turn both the apps on simultaneously so that the records could be matched on timestamps. We tested out this methodology with 10 minutes of data collection on Sep 8th, 2021, and found it to give successful record matching, after which the collection methodology was finalized.