Somewhat out of necessity, I develop software with my locale set to either "C" or "en_US". It's difficult to use a different locale because I only speak one language with anything even remotely approaching fluency.
As a result, I often overlook the differences in behavior that can be introduced by having different locale settings.Unsurprisingly, overlooking those differences will sometimes lead to bugs which are only discovered by some unfortunate user using a different locale. In particularly bad cases, that user may not even share a language with me, making the bug reporting process a challenging one. And, importantly, a lot of my software is in the form of libraries; while almost none of it sets the locale, it may be combined with another library, or used in an application which does set the locale - generating behavior I never experience myself.
To be a bit more specific, the kinds of bugs I have in mind are not missing text localizations or bugs in the code for using those localizations. Instead, I mean bugs where a locale changes the result of some locale-aware API (for example, toupper(3)
) when the code using that API did not anticipate the possibility of such a change (eg, in the Turkish locale, toupper
does not change "i" to "I" - potentially a problem for a network server trying to speak a particular network protocol to another host).
A few examples of such bugs in software I maintain:
- AttributeError in a Turkish locale
- imap relies on a C locale for date formatting
- Fix for locale-dependant date formatting in imap and conch
In the past, one approach I've taken to dealing with this is to write regression tests which explicitly change the locale to one where code was known not to work, exercise the code, verify correct behavior, and then restore the original locale. This works well enough, but only after someone has reported a bug, and it only covers one small area of a codebase.
Another approach which seems possible is to have a continuous integration system (CIS) set up to run a full suite of tests in an environment with a different locale set. This improves the situation somewhat, by giving as much coverage in that one alternate locale as the test suite normally gives. Another shortcoming is that there are many, many, many locales, and each may possibly cause different problems. In practice, there are probably only a dozen or so different ways a locale can break a program, but having dozens of extra testing configurations is taxing on resources (particularly for a project already stretching its resource limits by testing on different platforms, against different library versions, etc).
Another approach which occurred to me is to use (possibly first creating) a new locale which is radically different from the "C" locale in every way it can be - have a different case mapping, use a different thousands separator, format dates differently, etc. This locale could be used with one extra CIS configuration and hopefully relied upon to catch any errors in the code that could be triggered by any locale.
Does such a testing locale exist already? Are there flaws with this idea to testing for locale compatibility?
What other approaches to locale testing have people taken?
I'm primarily interested in POSIX locales, since those are the ones I know about. However, I know that Windows also has some similar features, so extra information (perhaps with more background information about how those features work), could perhaps also be useful.