Data syncing process to Central server using syncthing

SoP - periodic data syncing:

We have installed CLIx platform 2018 in clix school intervention school for rollouts,there are important responsibilities that we have such as 

• keeping the lab functioning
• enabling and following up with schools for rollout, supporting them
• making the overall system sustainable upto maximum possible extent.
• periodic data collection

Data collection is one of the major responsibilities of the CLIx team. For this to happen automatically (if the internet is available to CLIx school server machine) we have added a syncthing tool in the platform. The  primary objective of syncthing is to enable the thin data collection through the internet from the local server's (schools) to the central server (server in TISS, Mumbai data center) without any manual intervention. 

What is thin data? 

Thin data consists of  5 data points/sets which are given below:

• User Progress data - Which  will give us information with respect to which student has touched upon what activity in which subject. 
• Assessment (qbank) data - This contains student activity outputs of the assessment
• Tools’ log data - Raw data set that is being recorded for tools enabled with activity logging functionality.
• Activity timestamp - This data set will tell us about the amount of time students are spending in each activity.
• Server machines heartbeat log

Size of the thin data: The approximate monthly data size will be between 10-100MB on an average, which depends upon the rollout frequency, intensity and student activities and artifacts that gets generated, internet availability and frequency.

Keeping CLIx server ON for activity timestamp data: 

1. On the 2nd and 4th Friday of every month the school authorities (either teachers or the HM) will switch on the CLIx server machine in the computer lab so that the activity timestamp data files will get accumulated in a particular location on the local server( school server) . This process happens  without  the availability of the internet.
2. This process of activity timestamp data generation takes 3-5Hrs currently, which may go on higher side, depending upon the intensity as which rollout will happen and activities.
3. The authorities should start the CLIx server machine on suggested Fridays as soon as school starts and can shut down just before closing the school. This CLIx server machine should be started  and kept on the 2nd and 4th Fridays of the month even if there is no CLIx or any other lab session taking place on those days. Server machine should be kept ON and running for a minimum of 4 Hrs, beyond that is ideal (from starting till the end of the school day).
4. If the Internet is made available on the same day, then the syncthing will automatically sync the generated data with the central server (TISS). If not , whenever the internet is connected to the server, the syncing of the backed up data will happen.
5. Respective CLIx team members should send reminders about this, to their respective schools, before every 2nd and 4th Friday of the month.
6. Please note that, for the 2nd and 4th Friday, requirement is only to keep CLIx server machine ON and not all other machines, if those are not required.