When we try to select the ASCII name, the Unicode version also returns.
This statement is a bit of a misunderstanding of how encodings work. ASCII is an 8-bit encoding and a character set. It is values 0 - 127 and is common across most code pages and Unicode. However, it really only applies to VARCHAR
data. When using NVARCHAR
, then all characters are Unicode, even if those characters are found in other character sets. So here, you are only getting Unicode characters returned since NVARCHAR
only holds Unicode characters (encoded as UTF-16 Little Endian). It just so happens that the ASCII character set was duplicated as a subset of Unicode.
Meaning, what you are really saying here is that you want the regular Latin characters only, not the fullwidth version.
It seems that SQL Server maps Unicode based versions of each letter to their ASCII variant.
Yes and no. Windows and SQL Server can map Unicode characters to similar looking characters within an 8-bit code page, but that only happens when converting a Unicode string to an 8-bit code page (or from one code page to another). That is not happening here. Here, again, you are only dealing with Unicode. It just so happens that both regular and fullwidth forms of the US English alphabet are considered equal when the Collation is Width Insensitive. And based on your question and the test case (two separate things since a column's Collation is used when querying a column, but the DB's default Collation is used when dealing only with string literals and/or variables), it is clear that the Collations you are using (which could both be the same Collation) are Width Insensitive.
To fix this, please do not use a binary Collation. Using a binary Collation is the unfortunately commonly-accepted go-to answer to fix queries when people get more matches than they were expecting. And sometimes it is the correct answer, but more often than not, such as with this question, it isn't.
You simply need to add "width sensitivity" to the Collation that you are using. You can find the column's Collation with the following query, just fill in the correct table name and column name:
SELECT col.[collation_name]
FROM sys.columns col
WHERE col.[object_id] = OBJECT_ID(N'<schema_name>.<table_name>')
AND col.[name] = N'<column_name>';
If the Collation is a Windows Collation (i.e. the name does not start with SQL_
) then you might just be able to add _WS
to the end of the Collation name. For example:
Latin1_General_100_CS_AS
--> Latin1_General_100_CS_AS_WS
If the Collation is a SQL Server Collation (i.e. name does start with SQL_
), then none of those allow for width sensitivity and you should choose an equivalent Windows Collation. If the Collation is SQL_Latin1_General_CP1_*
, then try the same thing start with Latin1_General_100_
.
-- current Collation (no width sensitivity)
SELECT CASE WHEN N'Word' = N'Word' COLLATE Latin1_General_100_CI_AS THEN 1
ELSE 0 END;
-- 1
-- add width sensitivity
SELECT CASE WHEN N'Word' = N'Word' COLLATE Latin1_General_100_CI_AS_WS THEN 1
ELSE 0 END;
-- 0
-- confirm case INsensitivity
SELECT CASE WHEN N'WORD' = N'Word' COLLATE Latin1_General_100_CI_AS_WS THEN 1
ELSE 0 END;
-- 1
For more details on why you should first attempt to get the correct sensitivity before using a binary Collation, please see the following post of mine:
No, Binary Collations are not Case-Sensitive