0

I have a DataFrame with the results of a marathon race, where each row represents a runner and columns include data like "Start Time" (timedelta), "Net Time" (timedelta), and Place (int). A scatter plot of the start time vs net time makes it easy to visually identifiy the different starting corrals (heats) in the race:

Start time in seconds vs Net time in seconds

I'd like to analyze each heat separately, but I can't figure out how to divide them up. There are about 20,000 runners in the race. The start time spacings are not consistent, nor are the number of runners in a given corral

Gist of the code I'm using to organize the data: https://gist.github.com/kellbot/1bab3ae83d7b80ee382a

CSV with about 500 results: https://github.com/kellbot/raceresults/blob/master/Full/B.csv

2 Answers 2

2

There are lots of ways you can do this (including throwing scipy's k-means at it), but simple inspection makes it clear that there's at least 60 seconds between heats. So all we need to do is sort the start times, find the 60s gaps, and every time we find a gap assign a new heat number.

This can be done easily using the diff-compare-cumsum pattern:

starts = df["Start Time"].copy()
starts.sort()
dt = starts.diff()
heat = (dt > pd.Timedelta(seconds=60)).cumsum()
heat = heat.sort_index()

which correctly picks up the 16 (apparent) groups, here coloured by heat number:

example of groups coloured by heat

2
  • Thanks, this is just what I was looking for, and is very pretty to boot!
    – Kelly
    Jan 14, 2015 at 12:37
  • One last quick noob question... what's the syntax for coloring the heats like that?
    – Kelly
    Jan 16, 2015 at 2:37
2

If I understand correctly, you are asking for a way to algorithmically aggregate the Start Num values into different heats. This is a one dimensional classification/clustering problem.

A quick solution is to use one of the many Jenks natural breaks scripts. I have used drewda's version before:

https://gist.github.com/drewda/1299198

From inspection of the plot, we know there are 16 heats. So you can a priori select the number of classes to be 16.

k = jenks.getJenksBreaks(full['Start Num'].tolist(),16)
ax = full.plot(kind='scatter', x='Start Num', y='Net Time Sec', figsize=(15,15))
[plt.axvline(x) for x in k]

From your sample data, we see it does a pretty good job, but do the sparsity of observations fails to identify the break between the smallest Start Num bins:

enter image description here

1
  • Thinking back on it, DSM's solution is much better for this situation since the groups are deterministically separated. I have had trouble in the past using Scikit learn k-means one one dimensional data.
    – andrew
    Jan 15, 2015 at 0:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.