0

As I try to join tables together on a value that's represented in different data types, I get really odd errors. Please consider the following:

I have two tables; let's say one is in database "CoffeeWarehouse," and the other is in database "CoffeeAnalytics":

Table 1: CoffeeWarehouse.dbo.BeanInfo

Table 2: CoffeeAnalytics.dbo.BeanOrderRecord

Now, both tables have a field called OrderNumber (although in table 2, it's spelled as[order number]); in Table 1, it's represented as a string, and in Table 2, it's represented as a float.

I proceed to join the tables together:

SELECT ordernumber,
   bor.*
FROM CoffeeWarehouse.dbo.BeanInfo AS bni
     LEFT JOIN CoffeeAnalytics.dbo.BeanOrderRecord AS bor ON bor.[order number] = bni.ordernumber;

If I specify the order numbers I'd like by adding the following:

WHERE bni.ordernumber = '48911'

then I see the complete table I'd like- all the fields from the table I've joined are populated properly.

If I add more order numbers, it works too:

WHERE bni.ordernumber IN ('48911', '83716', '98811', ...)

Now for the problem:

Suppose I want to select everything in the table where another field, i.e. CountryOfOrigin, is not null. I'm not going to enter several thousand order numbers- I just want to use a where clause to weed out the rows with incomplete data.

So I add the following to my original query:

WHERE bor.CountryOfOrigin IS NOT NULL

When I execute, I get this error:

Msg 8114, Level 16, State 5, Line 1 Error converting data type varchar to float.

I get the same error if I even simply use this as a where clause:

WHERE bni.ordernumber IS NOT NULL

Why is this the case? When I specify the ordernumber, the join works well- when I want to select many ordernumbers, I get a conversion error.

Any help/insight?

11
  • 3
    I couldn't read past your OrderNumber being float. Why would that ever be the case?
    – sam yi
    Jan 15, 2018 at 18:47
  • @samyi The quick answer is "corporate bureaucracy." I would have stored it as a string.
    – daOnlyBG
    Jan 15, 2018 at 18:52
  • Try SELECT MAX(CAST(OrderNumber AS FLOAT)) FROM BeanInfo and you may find that you have strings that just don't convert to floats. Or perhaps SELECT TOP(1000) OrderNumber FROM BeanInfo ORDER BY OrderNumber DESC or SELECT * FROM BeanInfo WHERE IsNumeric(OrderNumber) = 0 to see if there are any in there? Then, either correct those values, or revert to explicit casting... ON CAST(bor.[order number] AS NVARCHAR(32)) = bni.ordernumber
    – MatBailie
    Jan 15, 2018 at 18:56
  • 1
    There must be a value in VARCHAR version of OrderNumber that is not convertible to FLOAT. Try explicitly converting float to varchar before doing the join. Either CTE, temp table, or subquery. Or all of the above.
    – sam yi
    Jan 15, 2018 at 19:00
  • 2
    An Integer would be like... a million times better choice then a float. Who the hell would choose a float? AHHH!
    – JNevill
    Jan 15, 2018 at 19:03

1 Answer 1

3

The SQL Server query optimiser can choose different paths to get your results, even with the same query from minute to minute.

In this query, say:

SELECT ordernumber,
   bor.*
FROM CoffeeWarehouse.dbo.BeanInfo AS bni
     LEFT JOIN CoffeeAnalytics.dbo.BeanOrderRecord AS bor ON bor.[order number] = bni.ordernumber
WHERE bni.ordernumber = '48911';

The query optimiser may, for example, take one of two paths:

  • It may choose to use BeanInfo as the "driving" table, use an index to narrow down the rows in that table to, say, a single row with order number 48911, and then join to BeanOrderRecord using just that one order number.
  • It may choose to use BeanOrderRecord as the driving table, join the two tables together by order number to get a full set of results, and then filter that resultset by the order number.

Which path the query optimiser takes will depend on a variety of things, including defined indexes, the number of rows in the table, cardinality, and so on.

Now, if it just so happens that one of your order numbers isn't convertible to a float—say someone typed '!2345' by accident—the first optimiser choice may always work, and the second one may always fail. But you don't get to choose which path the optimiser takes.

This is why you're seeing what you think of as weird results. In one of your queries, all the order numbers are being analysed and that's triggering the error, in another only order numbers that are convertible to float are being analysed, so there's no error. But it's basically just luck that it's working out the way it is. It could just as well be the other way around, or neither query might ever work.

This is one reason it's bad to store things in inappropriate data types. Fixing that would be the obvious solution.

A dirty and terrible fix, however, might be to always cast your FLOAT to a VARCHAR when doing the order number comparison, as I believe it's always safe to cast from FLOAT to VARCHAR. Though you may need to experiment to make sure the resulting VARCHAR value is formatted the same as your order number (or cast to INTEGER first...)

You'll have to resort to some quite fiddly trickery to get any performance out of your existing setup, though. If they were both VARCHAR values you could easily make the table join very fast by indexing each order number column, but as it is the casting you'll have to do will render normal indexes unusable for a join.

If you're using a recent version of SQL Server, you can use TRY_CAST to find the problem row(s):

SELECT * FROM BeanOrderRecord WHERE TRY_CAST([order number] AS VARCHAR) IS NULL

...will find rows with any FLOAT [order number] which can't be converted to a VARCAHR.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.