17

I have an sqlite database with over 400k records. I have just found that some of the text fields have carriage returns in them and I wanted to clean them out. I wanted to copy the structure of the original table and then do something like:

INSERT INTO large_table_copy 
SELECT date, other_fields, replace(dirty_text_field,XXX,"") 
FROM large_table

Where XXX is whatever the code would be for a carriage return. It's not \n. But I can't find out what it is.

3 Answers 3

33

SQLite lets you put line breaks inside string literals, like this:

SELECT replace(dirty_text_field, '
', '');

If you don't like this syntax, you can pass the string as a BLOB: X'0D' for \r or X'0A' for \n (assuming the default UTF-8 encoding).

Edit: Since this answer was originally written, SQLite has added a CHAR function. So you can now write CHAR(13) for \r or CHAR(10) for \n, which will work whether your database is encoded in UTF-8 or UTF-16.

4
  • 2
    For anyone who's not clear about this... you can do something like SELECT replace(dirty_text_field, X'0A', '\n')... to replace single \n chars with equivalent "\n" 2-char strings
    – Mark
    Sep 7, 2014 at 6:14
  • @MarkCarter this is a good answer, so why not post it answer instead of just as a comment?
    – aculich
    Dec 3, 2017 at 12:36
  • @aculich because it does not offer anything fundamentally new provided in the answer
    – Mark
    Dec 4, 2017 at 6:23
  • I don't think this works nowadays. sqlite> SELECT replace(whitelist.wl_date, CHAR(13), ''); results in Error: no such column: whitelist.wl_date. I've tried using database name as a prefix, and omitting the table name.
    – jww
    Sep 4, 2019 at 23:32
9

From @MarkCarter's comment on the question above:

SELECT replace(dirty_text_field, X'0A', '\n');
1

If you operate in a mixed environment with Windows/Linux/Mac it might be a safer solution to do a two step convertion:

SELECT replace( replace( dirty_text_field, CHAR(10), "<BR>"), CHAR(13), "") 
FROM temp_config;

This will work with most combinations of '\n' and '\r'

Your Answer

Reminder: Answers generated by Artificial Intelligence tools are not allowed on Stack Overflow. Learn more

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.