Bill Shupp Software engineer, photographer, musician, space geek

25Sep/1118

Speeding up satellite predictions in PHP with HipHop

ISS flyover data plotted on Google maps

Background

Seeing something in space fly overhead with the naked eye is pretty awesome.  It's not only a technical marvel to enjoy, but for me it helps keep things in perspective and my thoughts on the big picture.  Ever since the first time I saw the International Space Station fly by, I've felt that it is something everyone should experience.   I started looking at existing software to help remind me (and others) when the next pass of the ISS would be, and while there are a few good websites out there for satellite prediction, none of them did exactly what I wanted.  So a few months ago I decided to try and build my own.  The biggest piece of my project was going to be getting the actual satellite data.  So I looked around for existing data APIs that I could use, but couldn't find anything other than scraping existing sites (which is not only fragile, but some have no published data policy).  So, the first step was to figure out how to to generate the predictions myself.

How satellite predictions work

The process involves calculating the coordinates (position and velocity) of the observer, the satellite, and the sun at a given point in time using a common coordinate system called Earth Centered Intertial (ECI).  Then, you need to convert those coordinates to a topocentric-horizon or azimuth-elevation coordinate system that can be used to pinpoint the object in the sky.  But before you can do this, you first need to know the orbital path of the satellite.  NORAD, which has responsibility for tracking all man-made objects in orbit around Earth, uses the the Space Surveillance Network for collecting observational data of these objects.  It then distributes these observations as Keplerian orbital elements in the  Two Line Element Set (TLE) format.    (You can download these very small TLE files through sites like Celestrak).  The TLE data can then be used to  predict a satellite's position and velocity using Simplified Perturbations Models (SGP/SGP4/SDP4).  (NOTE: Since these models trade computational speed for accuracy, you need to update your TLE files every few days to make sure your predictions are accurate).  Dr. T.S. Kelso, who runs Celestrak, has a great series of columns  explaining all of this in detail.  I highly recommend them.

The library and performance problems

I couldn't find a suitable  or mature library in python or PHP implementing the SGP models, so I decided to port code from Gpredict, a real time satellite tracker application written in C.  Gpredict had almost everything I needed, the SGP model implementation, the prediction logic, etc..  The only thing I needed to add was the ability to determine the apparent magnitude (brightness) of an object, and I'd be set.  Though it took a few weeks, I finally got a web friendly PHP port of Gpredict working that produced prediction data which matched not only what the Gpredict application produced, but what other websites produced as well (including apparent magnitude for the ISS that I added).  While generating a snapshot or single pass was fast enough for my needs (under 50 milliseconds), generating predictions for the next 10 days was excruciatingly slow - in the order of several seconds.  After doing some profiling, I found there really wasn't much left to optimize.  PHP just wasn't fast at doing a LOT of math.  So this left me with a few options:  put the code into a php extension (which would be a fair amount of work), try a different language implementation of the SGP models (such as porting the Gpredict code to a web service, or creating a web service out of existing java code), or try Facebook's PHP to C++ transformation tool, HipHop for PHP.

HipHop for PHP to the rescue

I decided to try HipHop first since it would likely be the least amount of work and accomplish the biggest performance boost.  And though setting it up took some time to get right, the difference was very impressive.  A 10 day prediction that took 1866 milliseconds on my macbook pro in PHP took only 221 milliseconds when transformed and compiled with HipHop.  That's 88% less time!  Initially I was concerned that, since HipHop does not yet support PHP 5.3, I might have to do some refactoring.  But luckily only a few lines needed changing.  Further, HipHop does a great job of pointing out problems in the code.  Code that it doesn't understand (such as newer PHP 5.3 only code), as well as code that does not execute, or other general problems, are clearly logged during compile time.  It found a couple of bugs from my port of Gpredict that I hadn't run into yet, which was really helpful.  But as I mentioned before, HipHop for PHP is not terribly easy to set up.  It took me several hours from start to finish to get it installed and a program compiled and working.  Here's a quick list of things I ran into and you might want to watch out for:

  • HipHop for PHP doesn't work on Mac OS X, only Linux or FreeBSD.  If you run a mac, install Linux via VirtualBox or another VM tool and work there.
  • It won't compile on 32 bit Linux, only 64 bit is supported.
  • The README file is not up to date (it even says so).   Use the wiki pages on github instead.
  • The wiki includes some installation instructions for common distributions.  Do yourself a favor and use them.  There are also wiki pages covering how to use HipHop, which you'll need to read as well.
  • Make sure you have sufficient memory if you're running your compiled php program as a daemon.  I was using a small 512MB VPS, and it would routinely crash until I upped the memory to 1GB.

Had I known the above tips in advance, I probably could have done my install and had a program up and running in less than half the time that it took me.  But still, I'm pretty impressed with HipHop.  I have it running as a daemon locally, and make HTTP requests to it only for the long running prediction processes.  One cool thing is that I can return serialized php from the HipHop web service, deserialize the results in the client PHP process, and continue using those objects.  It made moving the predictions process from local to HipHop  a trivial change in my existing application.

Conclusion

Learning about how satellite tracking works and how to track them myself has been a blast.  But running into such a performance problem after spending time porting the prediction code to PHP was a disappointment.  However, HipHop for PHP was a perfect fit for solving this, and I'm happy with how well it works for my needs.

The application I set out to build isn't public just yet, but I did put up a prototype of real time ISS tracking using my newly ported Predict library, which you can view here.  Since I was new to satellite tracking, I needed some help understanding what needed to be done.  I would like to thank a few people who graciously lent their time to this process: David Lowe for helping to steer me in the right direction from the beginning (and putting up with a lot of questions), Dan Copeland for helping work through the math for calculating apparent magnitude as well as being a helpful second set of eyes, and T.S. Kelso for not only running Celestrak, but for being responsive and helpful in answering my questions and pointing me to additional information on the topic.

Cheers!

UPDATE: The application referenced in this article is now public.  Go to wheretheiss.at to try it out!

Comments (18) Trackbacks (1)
  1. Can you explain this further -> “One cool thing is that I can return serialized php from the HipHop web service, deserialize the results in the client PHP process, and continue using those objects. It made moving the predictions process from local to HipHop a trivial change in my existing application”

    I need a little how to pls

  2. The connection to the HipHop instance is over HTTP. So, in the php file run by HipHop, just do something like “echo serialize($myObject);” to send your object back serialized. In the call to the HipHop server, you would do something like: “$results = file_get_contents(‘http://myhiphopserver/file.php’); $myObject = unserialize($results);”.

  3. It’s very useful. I liked it. Thank you shupp. I’m going to set a virtual box than hiphop&php to have a test.

  4. I also recommend you to use cURL. I’ve read it’s faster compared to file_get_contents().

  5. My comment above which used file_get_contents() was simply an illustration, not a performance recommendation.

  6. its very useful. but i have doubt the implementation of sgp. how can you implement the sgp model.

  7. From the Predict README: “Predict is a partial PHP port of the Gpredict program that allows real-time tracking and orbit prediction of satellites from two line element sets. It supports the SGP4 and SDP4 models for prediction.” The code is is derived from the original code from the spacetrak #3 paper (http://celestrak.com/publications/AIAA/2006-6753/). I’ve confirmed that both Gpredict and Predict work for accurately predicting satellite passes. What is the source of your doubt?

  8. Thanks for the code! Very useful, but I need help understanding it: how can I retrieve the actual position (Lat, Lon,Alt) of a satellite (or the position at a given time) to do as in your example of the ISS?
    Thanks

  9. Sure. Check out this example here: https://gist.github.com/1741086 You’ll need to download the stations.txt from Celestrak (in the comments), and copy the first 3 lines to iss.tle, then just run this code once you have the Predict package in your include path.

  10. Thanks again, I will try it asap.

  11. Hi – basically what you are doing is similar to what I want to do, except I want to create it in an Android app. I’ve already worked through TLE files and such but could you tell me if you came across or know of anyway to calculate the latitude, longitude and velocity yourself without using Predict?

    Thanks

  12. You need the SGP models implemented. The Predict PHP code that implements them is ported from GPredict, which is written in C. Those routines are ultimately derived from libraries written by T.S. Kelso and others, you can find more about them on Celestrak.com. Before settling on GPredict as my port source, I did play with the AIAA-2006-6753 libraries, which includes a Java implementation that might work for you for Android. You can find my light modifications for it here https://github.com/shupp/sgp4. Hope that helps!

  13. Hello, the code is fantastic. If you are interested you can view what I used it for on this site:
    http://areeweb.polito.it/ricerca/E-STAR/ in the Live Tracking -> Current Position section (from left menu)
    There is only a little problem: the calculated height is very different from this code and the one given (same TLE) with STK or other softwares (everything else is identical)
    I am checking all the conversions but there is really a little to touch.
    Thanks again

  14. Gilbert: When testing the ISS, I compared it with GPredict, and the data was identical. Could you provide me some examples? (feel free to email me directly or post to the Predict github issue list)

  15. I have written a mail with 2 screens of gpredict and the website with your code.
    Thanks

  16. You are absolutely right, your code works fine but when I use the one implemented on my website (the same code with a little more parts to implement other things) it gives a difference.
    I probably have done some errors of implementation…
    I will check again.
    Thanks and sorry for the trouble.

  17. Solved, it was caused by a overwriting of some variables… stupid error I know… happens.
    Thanks again

  18. Gilbert: glad to hear. It’s nice to see Predict in the wild!


Leave a comment