Saturday, April 20, 2013

Connecting to the Polar Personal Trainer Site

Connecting to the Polar Personal Trainer Site

A number of people have requested access to the comms code that I use in the Polar HRM Uploader tool I created.  Because of this, I've decided to write about the software here and offer it free for non-commercial use.  It's not a complete implementation, you will still need to write your own HRM readers and convert that to the Polar format, but the attached code should give you a good starting point.

The communications protocol has been reversed engineered by intercepting the communications between Polar's WebLink and the Polar Personal Trainer Site.  I used an intercepting proxy, called BurpProxy, that does a great job of displaying the content of HTTP traffic.  Luckily, the Polar protocol is in plain text so it makes it relatively straightforward to determine what is required.

Click here to download all the source files in one archive. The code is also on GitHub

The Communications Protocol

Uploading to the Polar Site is a three stage process:
  1. Login.  This stage requires that you authenticate with the Polar site using a basic-auth method.  You can see an example of the login handshake here.
  2. Upload the timestamps of HRM files.  The Polar site will then validate the files and return a response which indicates which files have already been uploaded.  You can see an example of the timestamp handshake here.
  3. Upload the files.  The files are uploaded using a simple XML format.  You can see an example of uploading files here.

XML Marshalling

Since we need to convert from Java to XML, we use an XML data binding library called Castor.  Java has come along way since I originally wrote this code and you can now use Java's JAXB reference implementation to do something similar.

There's a number of good examples on the Castor site that shows how to use the libraries.

Click the link to get the Castor mapping file that I've used.

The code below shows a snippet of the XML format used by Polar.

 <wristunit-data>  
     <prop name="subModel">166</prop>  
     <prop name="version">2</prop>  
     <object name="userSettings">  
         <prop name="maxhr">186</prop>  
         <prop name="resthr">50</prop>  
     </object>  
     <collection name="Exercises">  
         <item type="Exercise">  
             <prop name="origin">1</prop>  
             <prop name="time">2011-08-22 17:39:52.000</prop>  
             <prop name="save">True</prop>  
             <object name="mode">  
                 <prop name="speed">True</prop>  
                 <prop name="cadence">True</prop>  
             </object>  
 ...              
   

We represent each element with a Java object:
  • Property.  These are the core elements on the Polar message that describe a name and value.  They have an optional type.
  • Item. An Item is a collection of Polar objects and it has a Type and an optional Index.
  • Collection.  A named list of Items, e.g. exercises.
  • Polar Object.  A named collection of Properties.
  • Wrist Unit Data. A wrapper object for the message.
  • Webservice.  A wrapper object for the Polar response.
You can download these files by clicking on the links or view them on GitHub.

Communications Code

The communications code is based upon standard Java HTTPUrlConnection and is contained in the class PolarSiteConnection.  This contains simple methods for connecting to the Polar site, sending data and parsing the response.

It encapsulates the Castor data binding too, which is adequate for my project but not generic enough as there's no easy means of implementing an alternative library without some refactoring of the code.

Here's a simple example of how to use the code to login:

 private void processLogin() {  
         // Update the status on the main dialog.  
         controlledFrame.setStatusMessage("Logging into Polar website...");  
         // Get the dialog credentials and try to connect.  
         String username = loginDialog.getUsername();  
         String password = loginDialog.getPassword();  
         System.out.println("Site connection is: " + siteConn);  
         // Pass these to the connection class.  
 ****  
         Webservice ws = siteConn.intiateConnection(username, password);  
 ****  
         if (ws == null) {  
             controlledFrame.setStatusMessage("Cannot connect to Polar site");  
         } else {  
             Status s = ws.getStatus();  
             if (s.isOK()) {  
                 userPreferences = ws.getUserPreferences();  
                 controlledFrame  
                         .setStatusMessage("Logged into Polar website as user "  
                                 + userPreferences.getFirstName() + " "  
                                 + userPreferences.getLastName());  
                 setAppState(AppState.LOGGED_IN);  
             } else {  
                 controlledFrame  
                         .setStatusMessage("Cannot log into the Polar website.");  
             }  
         }  
     }  

Note that the variable siteConn is an instance of PolarSiteConnection. The lines wrapped in **** are the interesting bits that send data to Polar.

PolarSiteConnection also uses a library to convert the password to Base64 as part of the Basic-Auth authentication.  You'll need to source your own copy of this as the licensing around the one I use prevents me from distributing it.





13 comments:

Unknown said...

Hi Curly I was wondering whether you might want to help me... I'm writing an application in C# to keep track of training and want to automatically read data that has been synchronized from Polar watches with polarpersonaltrainer.com. As a start, I'm trying to get your source code to run but keep getting 401 responses when trying to open the InputStream from the connection. However, when launching your app to upload HRM data I can successfully connect to PPT. When opening the URL (https://www.polarpersonaltrainer.com/weblink2/check.jsp) in a browser I can also log in using my usual credentials. I'm running Java 1.7.

Do you have an idea what might cause the problem? Thanks in advance!

Curly said...

Fabian, not sure what the problem is, other than the HTTP request is not being authorized correctly. I'm not familiar with C#, but you need to make sure that your SSL context is correctly configured. Have a close look at the relevant code in the HRMUploader here https://github.com/1ka/HRMUploader/blob/master/com/softwyer/hrmuploader/PolarSiteConnection.java#L168

Unknown said...

Thanks for pointing me the right direction. Apparently, the implementation of Base64 that I downloaded was not working as required. Using "DatatypeConverter.printBase64Binary(...)" does seem to work though.

However, instantiating the Unmarshaller on line 402 produces PropertiesException ("Could not instantiate configured class: org.castor.mapping.loaderFactories[0]=org.castor.mapping.JDOMappingLoaderFactory" followed by a few SecurityExceptions ("sealing violation: package org.castor.mapping is sealed"). I can successfully create the castor mapping. Do you have an idea what that might be about?

Curly said...

I'd hazard a guess that JDOMappingLoaderFactory isn't there or one of its dependencies is missing, perhaps one of the classes you are trying to map to? You might have to hunt around the Castor site for details on where it expects files to be.

A google search on the error message shows this link http://jira.codehaus.org/secure/attachment/28617/patch-C1927-20070725-01.txt which might contain a clue.

Unknown said...

Thanks again. I eventually managed to connect by using an older Castor-Version (1.1.1 instead of 1.3.3) and adding xercesImpl.jar.

However I'm still unable to retrieve my data from PPT. I've tried using your getConnection() to connect to links containing my training entries (or any other site on PPT after logging in) but I keep getting 404 (or 301 or 302 (redirect)) responses, probably to the PPT login site (which is what I get when opening the URL after logging out). Is there a way to use the established, authenticated connection to PATH_INITIALISE and "redirect" it to the training entry?

I don't think Polar WebSync normally downloads training entries from PPT onto the watch but maybe you still know the protocol required to retrieve the entries via XML...?

Curly said...

Fabian, I've not tried to download data from PPT. As part of the initial handshaking, PPT tells us what files have already been uploaded and that's it.

You might need to spoof the 'export' function off the website if you want to the data. I can see the following request to PPT when the button is clicked:

Remote Address:194....
Request URL:https://www.polarpersonaltrainer.com/user/calendar/index.jxml
Request Method:POST
Status Code:200 OK
Request Headersview source
Accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Accept-Encoding:gzip,deflate,sdch
Accept-Language:en-GB,en;q=0.8
Cache-Control:max-age=0
Connection:keep-alive
Content-Length:216
Content-Type:application/x-www-form-urlencoded
Cookie:locale=en; JSESSIONID=caaGOBIGipServerpool_polar_electro_http_personaltrainer=3255347210.36895.0000; calendar.type=list; calendar_start_date=2013-12-01; calendar_end_date=2014-04-07; ch=; old_ppt_user=true; activityfeed.items=target.result.message.fitnessdata.challenge.; __utma=7071310504637.2; __utmb=707104637; __utmc=70713105; __utmz=707131099404637.2.2.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=(not%20provided); __atuvc=7%7C16%7C19
DNT:1
Host:www.polarpersonaltrainer.com
Origin:https://www.polarpersonaltrainer.com
Referer:https://www.polarpersonaltrainer.com/user/calendar/item/analyze.ftl?id=270529223&r
User-Agent:Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/34.0.1847.131 Safari/537.36
Form Dataview sourceview URL encoded
.action:export
items.0.item:270529223
items.0.itemType:OptimizedExercise
.filename:CurlyWurly_06.05.2014_export.xml
items.0.item:270529223
items.0.itemType:OptimizedExercise
.filename:CurlyWurly_06.05.2014_export.xml

Unknown said...

Thanks again for your time and effort. I didn't know this export function existed, since I use the week view!

It looks like my knowledge of these Internet technologies is by far not extensive enough. I would be very grateful if you could help me out:

I saw in the HTML code that the Export-Button calls "exportSelectedCalItem('index.jxml?.action=export','mystyfly_09.05.2014_export.xml');" however I cannot call that method somehow, can I? Connecting to "https://www.polarpersonaltrainer.com/user/calendar/index.jxml?.action=export" doesn't work either. How would I be able to retrieve my data?

Curly said...

Fabian, if you stick BurpProxy onto your PC and connect to PPT through the proxy, you can see the request and response headers.

There's a whole bunch of cookie and param information sent with the POST. Not sure if the cookies are needed, but if so, you would need to capture them on the initial connection. I see the following:

Cookie locale en
Cookie ch
Cookie JSESSIONID baafF91YQ2eh99UOx0Hxu
Cookie BIGipServerpool_polar_electro_http_personaltrainer 3238569994.36895.0000
Cookie __utma 70713105.1113819529.1399652973.1399652973.1399652973.1
Cookie __utmb 70713105.3.10.1399652973
Cookie __utmc 70713105
Cookie __utmz 70713105.1399652973.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none)
Cookie old_ppt_user true
Cookie activityfeed.items target.result.message.fitnessdata.challenge.
Body .action export
Body items.0.item 270529223
Body items.0.itemType OptimizedExercise
Body .filename CurlyWurly_09.05.2014_export.xml

It should be possible to do this through an application, but it might not be that easy...

DigitalNeural said...

Hi Curly.
I was wondering if you're still doing this.
I came across your app and I've tried to use it.
The app opens the file and logs onto the Polar Personal Trainer site but then hangs when doing the upload.
I'm hoping you can help.
Thank you.

Curly said...

@Unknown

I'll take a look and feedback via the comments. Curly.

Jazz said...

Hi Curly, Great app thanks I have been using for years but have started to have some trouble logging into Polar, i think they may be putting secure logins in place as its sporadic and now seems to not work at all. I love this app and use weekly. Dont suppose its something you could look at quickly just to see if there is anything that can be easily adapted. Much thanks Jazz

Jazz said...

P.S. Thanks for a great tool

Curly said...

@Jazz Had a quick look but I'm struggling to even run this now. I had to migrate to an Amazon JDK due to Sun's fucking stupid licensing around Java (what a bunch of arseholes Oracle are, they've ruined everything they've touched). However, I can't seem to resolve the SSL connection issues with this Amazon JDK.

I did note this Polar communique though -> https://www.polar.com/uk-en/pptclosing which makes me think it's not worth even trying now. I'd migrated to Strava a while back, and would suggest you do the same, or to another site that isn't product related. It's not just Polar, Suunto are closing down their training site too (and pushing people to use phone App versionss - ugh!)

Anyway, ranting over :-)

Post a Comment