9

I have an entity collection of Readings. Each Reading is linked to an entity called Meter. (And each Meter holds multiple readings). each Reading holds a field for meter id (int) and a field for time.

Here is some simplified code to demonstrate it:

public class Reading
{
    int Id;
    int meterId;
    DateTime time;
}

public class Meter
{
    int id;
    ICollection<Readings> readings;    
}

Given a specific period and list of meterids, what would be the most efficient way to get for each Meter the first and last reading in that time period?

I am able to iterate through all meters and for each meter to obatin first and last reading for the period, but I was wandering if there is a more efficient way to acheive this.

And a bonus question: same question, but with multiple periods of time to get data for, instead of just one period.

2
  • Have you tried Queryable.First() and Queryable.Last() ?
    – user1914530
    May 27, 2013 at 6:26
  • Yes ,my solution is for every meter and period to take First() and Last() - but that doesn't take into account the fact that for all meters I'm looking at the same time period. maybe some sort of grouping would be more efficient here? May 27, 2013 at 6:29

7 Answers 7

3

I am not exactly sure how you want this data, but you could project it into an anonymous type:

var metersFirstAndLastReading = meters.Select(m => new 
    {
        Meter = m,
        FirstReading = m.readings.OrderBy(r => r.time).First(),
        LastReading = m.readings.OrderBy(r => r.time).Last()
    });

You can then read your result list like this (this example is just meant as an illustration):

foreach(var currentReading in metersFirstAndLastReading)
{
    string printReadings = String.Format("Meter id {0}, First = {1}, Last = {2}", 
                               currentReading.Meter.id.ToString(),
                               currentReading.FirstReading.time.ToString(),
                               currentReading.LastReading.time.ToString());

    // Do something...
}

Another option would be to create properties in Meter which dynamically return the first and last readings:

public class Meter
{
    public int id;
    public List<Reading> readings;

    public Reading FirstReading 
    {
        get
        {
            return readings.OrderBy(r => r.time).First();
        }
    }

    public Reading LastReading
    {
        get
        {
            return readings.OrderBy(r => r.time).Last();
        }
    }
}

EDIT: I misunderstood the question a little.

Here is the implementation to determine the first and last readings for a meter including a date range (assuming meterIdList is an ICollection<int> of IDs and begin and end is the specified date range)

var metersFirstAndLastReading = meters
    .Where(m => meterIdList.Contains(m.id))
    .Select(m => new 
    {
        Meter = m,
        FirstReading = m.readings
                        .Where(r => r.time >= begin && r.time <= end)
                        .OrderBy(r => r.time)
                        .FirstOrDefault(),
        LastReading = m.readings
                        .Where(r => r.time >= begin && r.time <= end)
                        .OrderByDescending(r => r.time)
                        .FirstOrDefault()
    });

You won't be able to use properties now (as you need to supply parameters) so methods will work just fine as an alternative:

public class Meter
{
    public int id;
    public List<Reading> readings;

    public Reading GetFirstReading(DateTime begin, DateTime end)
    {
        var filteredReadings = readings.Where(r => r.time >= begin && r.time <= end);

        if(!HasReadings(begin, end))
        {
            throw new ArgumentOutOfRangeException("No readings available during this period");
        }

        return filteredReadings.OrderBy(r => r.time).First();
    }

    public Reading GetLastReading(DateTime begin, DateTime end)
    {
        var filteredReadings = readings.Where(r => r.time >= begin && r.time <= end);

        if(!HasReadings(begin, end))
        {
            throw new ArgumentOutOfRangeException("No readings available during this period");
        }

        return filteredReadings.OrderBy(r => r.time).Last();
    }

    public bool HasReadings(DateTime begin, DateTime end)
    {
        return readings.Any(r => r.time >= begin && r.time <= end);
    }
}
9
  • You need to add in 'where' clauses to take into account the time period
    – jammykam
    May 27, 2013 at 7:40
  • Readings belong to a Meter, so you don't need to perform any filtering.
    – Dave New
    May 27, 2013 at 7:44
  • @davenewza, thank you for your reply. filtering by time is required. but in any case, I'm not sure if this will increase performance. it still goes through every meter , and asks for first and last, right? May 27, 2013 at 8:49
  • 1
    @davenewza , I ended up using your solution with metersFirstAndLastReading (second one after the "misunderstood" pharse. It did help. if you would be so kind as to correct two things it, I will gldaly mark it as the answer. 1. Last() is not valid. instead , you need to order by descending and select First() (or better yet FirstOrDefault() ). 2. The filtering by meter ids is missing. I will add the code that worked to my question. Cheers :-) May 28, 2013 at 8:35
  • 1
    @omer: Updated code with a meterIdList and fixed the OrderByDescending part (oops!). Whether you want to use First() or FirstByDefault() depends on the nature of your data. You would definitely want your code to throw an exception when data is most certainly expected - this is up to you :)
    – Dave New
    May 28, 2013 at 8:53
1

I have a very similar data model where this code is used to get the oldest readings, i just changed it to also include the newest.

I use query syntax to do something like this:

var query = from reading in db.Readings
            group reading by reading.meterId
            into readingsPerMeter
            let oldestReadingPerMeter = readingsPerMeter.Min(g => g.time)
            let newestReadingPerMeter = readingsPerMeter.Max(g => g.time)
            from reading in readingsPerMeter
            where reading.time == oldestReadingPerMeter || reading.time == newestReadingPerMeter 
            select reading; //returns IQueryable<Reading> 

That would result in a only the newest and oldest reading for each meter.

The reason i think this is efficient is because its one lookup to the DB to get all the readings for each meter, instead of several lookups for each meter. We have ~40000 meters with ~30mil readings. i just tested the lookup on our data it took about 10s

The sql preformed is a crossjoin between two sub selects for each of the min and max dates.

UPDATE:

Since this is queryable you should be able to supply a period after, like this:

query.Where(r=>r.time > someTime1 && r.time < someTime2)

Or put it into the original query, i just like it seperated like this. The query isnt executed yet since we havent performed an action that fetches the data yet.

4
  • @Thanks , please note that: a. this does not filter by time. b. it returns either first or last reading , and not both. but I think I get the idea. May 27, 2013 at 9:34
  • Updated to add the period part, it will take the newest and oldest reading within the period for each meter.
    – Jim Wolff
    May 27, 2013 at 9:42
  • this does seem to improve performance. But for each meter it gives only one reading, not two. any idea on how to get both first ad last? Thanks May 27, 2013 at 11:11
  • 1
    Yes i can see the problem is that this returns one row with both the results, when translated into sql, instead of a row for each result (min and max)
    – Jim Wolff
    May 27, 2013 at 11:26
0

Create a new class as the return type called Result, which looks like this

public class Result
{
    public int MeterId;
    public Readings Start;
    public Readings Last;
}

I emulated your situation by making a list of Meters and populating some data, your query should be pretty much the same though

var reads = Meters.Where(x => x.readings != null)
                  .Select(x => new Result
                          {
                              MeterId = x.id,
                              Start = x.readings.Select(readings => readings).OrderBy(readings=>readings.time).FirstOrDefault(),
                              Last = x.readings.Select(readings=>readings).OrderByDescending(readings=>readings.time).FirstOrDefault()
                          });
2
  • Or OP could use anonymous types.
    – Dave New
    May 27, 2013 at 7:39
  • Yeah, I just prefer typed but anon is good too. I think using a return type adds a little clarity.
    – Lotok
    May 27, 2013 at 7:39
0
public IEnumerable<Reading> GetFirstAndLastInPeriod
    (IEnumerable<Reading> readings, DateTime begin, DateTime end)
{
    return
        from reading in readings
        let span = readings.Where(item => item.time >= begin && item.time <= end)
        where reading.time == span.Max(item => item.time) 
            || reading.time == span.Min(item => item.time)
        select reading;            
}
0
meters.Where(mt=>desiredMeters.Contains(mt)).Select(mt=>
   new{
     mt.Id,
     First = mt.Readings.Where(<is in period>).OrderBy(rd=>rd.Time).FirstOrDefault(),
     Last = mt.Readings.Where(<is in period>).OrderBy(rd=>rd.Time).LastOrDefault()
   });

If you have lots of readings per meter, this will not perform well, and you should consider Readings to be of SortedList class.

0

my solution will return exact what u want (List of all Meters containing Readings within given Time Period)

public IList<Reading[]> GetFirstAndLastReadings(List<Meter> meterList, DateTime start, DateTime end)
     {       
        IList<Reading[]> fAndlReadingsList = new List<Reading[]>();

            meterList.ForEach(x => x.readings.ForEach(y =>
            {
                var readingList = new List<Reading>();
                if (y.time >= startTime && y.time <= endTime)
                {
                      readingList.Add(y);
                      fAndlReadingsList.Add(new Reading[] { readingList.OrderBy(reading => reading.time).First(), readingList.OrderBy(reading => reading.time).Last() });
                }
            }));

       return fAndlReadingsList;
    }
0

I got some very nice leads, thank to all the responders. Here is the solution that worked for me:

        /// <summary>
        /// Fills the result data with meter readings matching the filters.
        /// only take first and last reading for each meter in period.
        /// </summary>
        /// <param name="intervals">time intervals</param>
        /// <param name="meterIds">list of meter ids.</param>
        /// <param name="result">foreach meter id , a list of relevant meter readings</param>
        private void AddFirstLastReadings(List<RangeFilter<DateTime>> intervals, List<int> meterIds, Dictionary<int, List<MeterReading>> result)
        {
            foreach (RangeFilter<DateTime> interval in intervals)
            {
                var metersFirstAndLastReading = m_context.Meter.Where(m => meterIds.Contains(m.Id)).Select(m => new
                {
                    MeterId = m.Id,
                    FirstReading = m.MeterReading
                                    .Where(r => r.TimeStampLocal >= interval.FromVal && r.TimeStampLocal < interval.ToVal)
                                    .OrderBy(r => r.TimeStampLocal)
                                    .FirstOrDefault(),
                    LastReading = m.MeterReading
                                    .Where(r => r.TimeStampLocal >= interval.FromVal && r.TimeStampLocal < interval.ToVal)
                                    .OrderByDescending(r => r.TimeStampLocal)
                                    .FirstOrDefault()
                });

                foreach (var firstLast in metersFirstAndLastReading)
                {
                    MeterReading firstReading = firstLast.FirstReading;
                    MeterReading lastReading = firstLast.LastReading;

                    if (firstReading != null)
                    {
                        result[firstLast.MeterId].Add(firstReading);
                    }

                    if (lastReading != null && lastReading != firstReading)
                    {
                        result[firstLast.MeterId].Add(lastReading);
                    }

                }

            }
        }


    }

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.