Clock Synchronization
The distributed computational capabilities of the vehicle require clocks on computers to be synchronized, in order for tf to be used to visualize and fuse sensor data. This page documents why (and gives the long story "how") to synchronize each computer on the network. Use Synchronize Clocks for the short story.
Overview
To utilize time sychronization, chrony must be installed. Chrony is a time synchronization utility that runs as a daemon process, and prevents clocks from drifting apart. Chrony also prevents clocks from jumping violently, so it works best when hardware clocks are already nearly synchronized.
Make sure rtd4.local has the "right" time
Our network setup is isolated most of the time, which means that individual machines on the network might have clock drift: but more dangerously, devices are usually not connected to mains power, so when switched off they sometimes revert back to Unix 0 time (aka, the Disco era) when Led Zeppelin was still touring and Y2K was too distant to think about.
To fix this, there are a few things you need to do on rtd4.local as soon as you Cold start the car.
- ssh into rtd4.local
- set the date to be something fairly authoritative, like the time on your phone:
sudo date -s "Sat Apr 22 14:39:00 MST 2017"
You'll need sudo privileges to do this and it will require your password, so you might have to type this a few times to get within a few seconds.
IF rtd4.local is connected to the internet somehow, you can run
sudo service ntp restart
Which should connect to a ubuntu ntp server to get its time. The chrony.conf for rtd4.local is:
driftfile /var/lib/chrony/chrony.drift # Comment this line out to turn off logging. log tracking measurements statistics logdir /var/log/chrony # Stop bad estimates upsetting machine clock. local stratum 8 manual # Allow computers on the unrouted nets to use the server. allow 192.168/16
Install chrony on your machine
If you want to visualize live data from the catvehicle's sensors, you will need to be synchronized to the clock on rtd4.local.
sudo apt-get install chrony
Synchronize your clock now
Once chrony is installed on your machine, you will need to edit your /etc/chrony/chrony.conf to look like the following:
# in /etc/chrony/chrony.conf on a client machine server rtd4.local iburst driftfile /var/lib/chrony/drift logdir /var/log/chrony log measurements statistics tracking
Important: don't run this next part if rtd4.local has a 1969 timestamp, or your machine will start marching backward in time. If rtd4.local has the right time, make sure chrony is running by restarting the daemon process:
sudo service chrony restart
How do I know it's working?
Check the logs to see what happens. Here's an example of how I started it on my machine on April 11. The machine began as 22.21 seconds off of the clock on rtd4.local. Over the next 4 seconds it checks the time several times, and then 30 seconds later the clock is within 1.8e-3 seconds of the time.
head /var/log/chrony/measurements.log ======================================================================================================================== Date (UTC) Time IP Address L St 1234 abc 5678 LP RP Score Offset Peer del. Peer disp. Root del. Root disp. ======================================================================================================================== 2017-04-11 19:48:54 192.168.0.102 N 10 1111 111 1111 6 6 0.00 2.221e+01 2.425e-02 4.965e-05 0.000e+00 0.000e+00 2017-04-11 19:48:56 192.168.0.102 N 10 1111 111 1111 6 6 0.00 2.221e+01 1.909e-02 3.931e-05 0.000e+00 0.000e+00 2017-04-11 19:48:58 192.168.0.102 N 10 1111 111 1111 6 6 0.00 2.221e+01 2.699e-02 5.512e-05 0.000e+00 0.000e+00 2017-04-11 19:49:22 192.168.0.102 N 10 1111 111 1111 6 6 0.00 1.833e-03 3.240e-02 6.597e-05 0.000e+00 0.000e+00 2017-04-11 19:50:22 192.168.0.102 N 10 1111 111 1111 6 6 0.00 2.372e-02 2.709e-02 5.535e-05 0.000e+00 0.000e+00
If you want to watch yours synchronize over a few minutes, try:
tail -f /var/log/chrony/measurements.log