6

We experienced a number of errors on our live application a week or two ago that have so far escaped explanation. We saw these errors internally and they were also experienced by clients as it manifested in a set of web services.

I have included the inner exception below, the project uses the CSLA framework and the error occured when retrieving an object from the database.

No known changes were made to the system at the time we started experiencing the errors, the infrastructure consists of a number of load balances web servers.

The errors seemed to be isolated to one of our servers, we experienced them using a console application connecting to the web services. The server in question was using a local DMZ IP to resolve the web services in its hosts file and by forcing this to go externally it seemed to resolve the issues.

It seems to be a very fine line between application and infrastructure to isolate this, so I'm wondering if anyone has any ideas or theories that could possibly explain this?

<InnerException>
      <ExceptionType>System.InvalidOperationException, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089</ExceptionType>
      <Message>Exception of type 'System.InvalidOperationException' was thrown.</Message>
      <Source>mscorlib</Source>
      <HelpLink />
      <Property name="Data">System.Collections.ListDictionaryInternal</Property>
      <Property name="TargetSite">Void VerifyIntegrity()</Property>
      <StackTrace>   at System.Runtime.CompilerServices.ConditionalWeakTable`2.VerifyIntegrity()
   at System.Runtime.CompilerServices.ConditionalWeakTable`2.Add(TKey key, TValue value)
   at System.Linq.Expressions.Expression..ctor(ExpressionType nodeType, Type type)
   at System.Data.Linq.SqlClient.Translator.TranslateLink(SqlLink link, List`1 keyExpressions, Boolean asExpression)
   at System.Data.Linq.SqlClient.SqlBinder.Visitor.ConvertToFetchedExpression(SqlNode node)
   at System.Data.Linq.SqlClient.SqlBinder.Visitor.ConvertLinks(SqlExpression node)
   at System.Data.Linq.SqlClient.SqlBinder.Visitor.FetchExpression(SqlExpression expr)
   at System.Data.Linq.SqlClient.SqlBinder.Visitor.VisitMember(SqlMember m)
   at System.Data.Linq.SqlClient.SqlVisitor.Visit(SqlNode node)
   at System.Data.Linq.SqlClient.SqlBinder.Visitor.VisitExpression(SqlExpression expr)
   at System.Data.Linq.SqlClient.SqlBinder.Visitor.VisitNew(SqlNew sox)
   at System.Data.Linq.SqlClient.SqlVisitor.Visit(SqlNode node)
   at System.Data.Linq.SqlClient.SqlBinder.Visitor.VisitExpression(SqlExpression expr)
   at System.Data.Linq.SqlClient.SqlBinder.Visitor.VisitSelect(SqlSelect select)
   at System.Data.Linq.SqlClient.SqlVisitor.Visit(SqlNode node)
   at System.Data.Linq.SqlClient.SqlBinder.Visitor.VisitAlias(SqlAlias a)
   at System.Data.Linq.SqlClient.SqlVisitor.Visit(SqlNode node)
   at System.Data.Linq.SqlClient.SqlVisitor.VisitSource(SqlSource source)
   at System.Data.Linq.SqlClient.SqlBinder.Visitor.VisitSelect(SqlSelect select)
   at System.Data.Linq.SqlClient.SqlVisitor.Visit(SqlNode node)
   at System.Data.Linq.SqlClient.SqlBinder.Visitor.VisitIncludeScope(SqlIncludeScope scope)
   at System.Data.Linq.SqlClient.SqlVisitor.Visit(SqlNode node)
   at System.Data.Linq.SqlClient.SqlBinder.Bind(SqlNode node)
   at System.Data.Linq.SqlClient.SqlProvider.BuildQuery(ResultShape resultShape, Type resultType, SqlNode node, ReadOnlyCollection`1 parentParameters, SqlNodeAnnotations annotations)
   at System.Data.Linq.SqlClient.SqlProvider.BuildQuery(Expression query, SqlNodeAnnotations annotations)
   at System.Data.Linq.SqlClient.SqlProvider.System.Data.Linq.Provider.IProvider.Execute(Expression query)
   at System.Data.Linq.DataQuery`1.System.Linq.IQueryProvider.Execute[S](Expression expression)
   at System.Linq.Queryable.SingleOrDefault[TSource](IQueryable`1 source)
   at NamespaceA.DaSql.NamespaceB.NamespaceBContext.NamespaceA.Da.NamespaceB.INamespaceBContext.GetClassA(Int32 objectId)
   at NamespaceA.NamespaceB.ClassA.DataPortal_Fetch(SingleCriteria`2 criteria)
   at dm(Object , Object[] )
   at Csla.Reflection.MethodCaller.CallMethod(Object obj, DynamicMethodHandle methodHandle, Object[] parameters)</StackTrace>
    </InnerException>

Thank you in advance for any help or theories.

Edit :

Full exception is here

LINQ to SQL below, nothing to it and ObjectA is simply a wrapper class with a properties. Nothing more than a simple select and populate of one object bases on an ID. ctx is a CSLA ContextManager.

var data = from d in ctx.DataContext.ObjectAs
                           where d.ObjectId == objectId
                           select new ObjectA
                            {
                                Id = d.DispatchId,
                                ClientId = d.ClientId,
                                DateCreated = d.DateCreated
                            };

                return data.SingleOrDefault();
8
  • Better add the calling code and outer exception as well.
    – ℍ ℍ
    Aug 22, 2011 at 16:09
  • The outer exception is massive, duplicates what is posted and provides no further information. The executing code is a very standard simple LINQ to SQL select. I will post it all up tomorrow.
    – rrrr-o
    Aug 22, 2011 at 16:37
  • More info added as requested.
    – rrrr-o
    Aug 23, 2011 at 9:15
  • AfaIct there is an "Invalid Operation" happening in the SingleOrDefault. That would mean a Db or WebService error, not further related to this code.
    – ℍ ℍ
    Aug 23, 2011 at 9:26
  • I would agree with that, but struggling to find any information on possible causes.
    – rrrr-o
    Aug 23, 2011 at 9:40

3 Answers 3

2

I see the inner exception you have attached and as per exception, may be there is a problem with your collection which is created when the SingleOrDefault() method gets executed (LINQ is deffered execution).

I am posting the 2 links which might help you to analyze the problem and solution better if you dig a little deep into the collection creation. Hope these links help you solve your problem:

Link1: http://typedescriptor.net/browse/members/289638-System.Runtime.CompilerServices.ConditionalWeakTable%602%5BTKey,TValue%5D.VerifyIntegrity()

Link 2: http://code.google.com/p/bclcontrib-abstract/source/browse/%2BFromCoreEx/%2BKludge/Runtime/CompilerServices/ConditionalWeakTable.cs?spec=svnd7b68d68e34be8e5db308feaccf935afd2c1b8d9&name=d7b68d68e3&r=d7b68d68e34be8e5db308feaccf935afd2c1b8d9

The culprit methods might be ConditionalWeakTable.Add() and ConditionalWeakTable.VerifyIntegrity()

Above pointers should help you reach to a solution. Thanks

5
  • Thanks for the links, this is info that I've already seen and one link is posted above on the question comments. The point of the question still remains that I cannot find an information on why this is happening. Why is the ConditionalWeakTable invalid and why would this persist until some internal routing was changed.
    – rrrr-o
    Oct 10, 2011 at 10:39
  • @rrrr according to the link, _invalid is set to true at the start of a lock, and back to false at the end of the lock. The idea is that any event that causes an unexpected unlock (like out of memory) will leave _invalid set, and InvalidOperationException would then be thrown on the next lock attempt (specifically at the VerifyIntegrity() calls). As the code explains: private bool _invalid; // flag detects if OOM or other background exception threw us out of the lock. Oct 10, 2011 at 17:12
  • The closest to an answer is "OOM or other background exception" however this raises more questions than it answers. I think I'm resigned to not knowing exactly what caused this, simply that it was an unexpected exception most probably at server level. If it reoccurs then I will have to get the infrastructure team to investigate more into server resource usage at the time.
    – rrrr-o
    Oct 11, 2011 at 9:41
  • Accepting this primarily for the second link in the answer and the "OOM or other background exception" comment in the source code.
    – rrrr-o
    Oct 17, 2011 at 10:21
  • Thanks @rrrr I hope your problem is solved and hope to meet you in future at SO..
    – S2S2
    Oct 17, 2011 at 15:38
0

I think that as illustrated in the msdn documentation an InvalidOperationException could be thrown if the method SingleOrDefault() is called in a sequence that contains more than one element, maybe you could use FirstOrDefault() instead

Hope this helps.

1
  • As much as you are technically correct that that an InvalidOperationException exception is thrown in the scenario you mentioned, it is in a completely different context and with a helpful error message "System.InvalidOperationException: Sequence contains more than one element" and the call stack is completely different. The key here is that the error seems to be happening within "ConditionalWeakTable".
    – rrrr-o
    Oct 5, 2011 at 8:32
0

I would put a breakpoint on the

return data.SingleOrDefault();

line. Then, mouse over data to evaluate it, and see if it is returning what you expect. It may show that an exception was thrown. If an exception is not being thrown, try evaluating data.SingleOrDefault() with either a mouse over or by adding a watch expression, and see if you get an exception.

You could also separate the query into a single piece for each step, and evaluate at each step to identify which of the calls is causing the exception.

Ex:

// break execution and evaluate each one separately
var table = ctx.DataContext.ObjectAs;
var filter = table.Where(x => x.ObjectId == objectId);
var select1 = filter.Select(x => x.DispatchId);
var select2 = filter.Select(x => x.ClientId);
var select3 = filter.Select(x => x.DateCreated);
var select4 = filter.Select(x => new ObjectA
                  {
                            Id = d.DispatchId,
                            ClientId = d.ClientId,
                            DateCreated = d.DateCreated
                  };
var singleOrDefault = select4.SingleOrDefault();

This will narrow down which part of your query is causing the issue.

1
  • We cannot replicate this, this has only happened twice in our live environment.
    – rrrr-o
    Oct 10, 2011 at 10:30

Your Answer

Reminder: Answers generated by Artificial Intelligence tools are not allowed on Stack Overflow. Learn more

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.