Questions tagged [utf-8]
UTF-8 is a character encoding that describes each Unicode code point using a byte sequence of one to four bytes. It is backwards-compatible with ASCII while still supporting representation of all Unicode code points.
21,845
questions
1
vote
1
answer
20
views
Cay anyone show me the calculation how utf-8 represents 1112064 characters?
I am not understanding how utf-8 represents 1112064 characters,
my calculation is something like this 2^7+2^11+2^16+2^21=2164864 characters as in utf-8, for 1 byte it has 7 bits, for 2 bytes it has 11 ...
0
votes
0
answers
11
views
Content-Type header does not indicate utf-8 encoded json: application/json
I'm trying to run a Rest API automation using webdriver/6.1.11 and Selenium Jar 4.0.0-alpha-1.
Browser Safari 15.3, Apple OS X
Until last week it was working fine, but now when I run, it's giving me ...
0
votes
0
answers
37
views
Why is flutter throwing an Bad UTF-8 encoding error when I get json data?
class _CalendarScreenState extends State<CalendarScreen> {
Future<void> calendarCheck() async {
http.Response response = await http
.get(Uri.parse('https://192....
0
votes
0
answers
31
views
R - intToUtf8 function
Using the gsub function, I am replacing certain string of texts that contain the Unicode Character 'START OF GUARDED AREA' (U+0096).
gsub("A string with the following character –","A ...
-1
votes
0
answers
28
views
Strange beginning characters in an XML response
We have been getting strange characters(three question marks - ???) in our Http response. I thought this might an encoding issues and be the BOM that is probably not detected, but it does not appear ...
0
votes
0
answers
40
views
Encoding problem: utf8mb4 Database with Latin1 to UTF-8 in PHP [duplicate]
I have MySQL database table "PRICES" that is set SET NAMES utf8mb4; with colum name "TYPE" char(70) CHARACTER SET latin1 COLLATE latin1_swedish_ci
When I get the data in PHP 5.6.40 ...
0
votes
0
answers
23
views
Import data from utf8mb4 Table to utf8 Table in MySql [duplicate]
i imported fields and data from a utf8mb4_unicode_520_ci table to a table with utf8_general_ci fields
But when I go to print I see all the strange characters (in php). How do I convert these fields ...
0
votes
1
answer
38
views
How to test the first characters of a file in php?
I have a PHP code that allows me to read a csv file, insert the data into the database and move this file to another folder once the processing is finished.
This code works by default with UTF8 BOM ...
0
votes
0
answers
18
views
Problem with opening UTF-8 encoded CSV files in Notepad or Microsoft Excel
I have a problem with opening UTF-8 encoded CSV files in Notepad or Microsoft Excel. On one computer works fine but on the other cannot decode letters properly.
These files contains some Serbian ...
1
vote
1
answer
30
views
Why is FedEx API returning encoded characters?
FedEx Address Validation API is returning encoded characters and a 400 bad request error. Here is my request body with the secret key blocked out for obvious reasons:
POST /address/v1/addresses/...
0
votes
1
answer
20
views
How to make search of list with none english letters?
I am making e-commerce website on ( React.js - client )and ( Python Django - client ). I am trying to make a search feature for list of all products and the queries are none-english. Whenever I try to ...
0
votes
0
answers
33
views
Decoding "unknown" strings to UTF-8 with Javascript
I have a text source that sends me encoded Strings, for the most part, the JavaScript function decodeURIComponent(escape(str)) works, But I've logged some weird "unknown" characters that ...
0
votes
0
answers
41
views
Yet another Powershell problem with UTF-8
Why do I manage to do this directly in the Powershel console :
PS C:\windows\system32> read-host -prompt "éàèçç ?"
éàèçç ?: éééééé
éééééé
But doing the exact same thing from a .ps1 ...
0
votes
1
answer
29
views
How to convert to UTF-8 in SQL Server database?
i'm working on mssql server 2017.
i got table with 'name' nvarchar column with values that look like '\u039b \u041e \u0422 \u0422 \u0410'
(sequence of '\u0[number][number][number or letter]')
how do i ...
0
votes
1
answer
34
views
How to set file encoding to ISO-8859-1 or WinCP-1252 in C++
I am learning C++ and I have a requirement to write a CSV file encoded in ISO-8859-1 or WinCP-1252.
I've tried the following code snippet to set a locale that will use 1252 codepage encoding, but when ...
0
votes
0
answers
34
views
Some characters in UTF-8 has the same value
I have a server that works with windows-1252 and I have a problem with characters à, À, í and Í. I need to replace this special characters to a, A, i and I in PHP.
As you can see in the following URL ...
2
votes
2
answers
24
views
Replace coded text by unicode text in Vietnamese
I have a csv file saved under the name as sample.csv as follows:
No,duong
1, Ðu<U+1EDD>ng ÐT 605
2, Ðu<U+1EDD>ng Nam K<U+1EF3> Kh<U+1EDF>i Nghia
3, Ðu<U+1EDD>ng Duy Tân
...
0
votes
0
answers
25
views
Python: How to prevent the implicit removal of diacritical marks from Unicode characters during keyboard input
I'm using Python 2.7 in Windows.
In the Python command-line, when I entered this:
u'À'
The output became this:
u'A'
Somehow, the diacritical marks of some characters are removed. How do I prevent ...
0
votes
0
answers
47
views
How to decrypt chrome cookies in python in linux
I want to decrypt chrome cookies.
chromium version - 103.0.5060.53
I tried the code in this stackoverflow post: Decrypting Chromium cookies
my code:
from Crypto.Cipher import AES
from Crypto.Protocol....
2
votes
0
answers
25
views
utf8 locale in boost::filesystem::path::imbue in DLL crashes application after unloading dll
Every char string in our codebase is utf8.
In order to properly convert these to utf16 strings within boost::filesystem::path a suitable locale is imbued. We're delivering a DLL for third party use. ...
0
votes
1
answer
24
views
Chinese conversion in MultiByteToWideChar
I'm trying to display a Chinese text in the MessageBoxW. But I can't correctly convert it from UTF-8 to wchar_t. At the same time, the original wchar_t Chinese is displayed correctly.
I played with ...
1
vote
0
answers
39
views
Laravel: How to save text file in UTF-8 without BOM
Problem: We need to save txt file in UTF8 without BOM. However, string with special chars like C'est français are getting saved directly into UTF8 without BOM. However, string without special chars ...
1
vote
0
answers
57
views
memory leak in utf8 string parser
typedef struct _utf8_multibyte {
uint8_t bytes[4]; /* always max of 4 */
int used; /* how many of the 4 bytes are used */
} utf8_multibyte_t;
typedef struct _utf8_string {
uint8_t *raw; /*...
-1
votes
0
answers
13
views
Problem in python with reading a file that contains german umlauts [duplicate]
let's say we have a textfile with the following content:
Öffnungszeiten
if i'm trying to read the file and print it out via
with open("textfile.txt", "r", encoding="utf-8/...
-2
votes
0
answers
22
views
Database problem with language even with UTF-8
I have problem with language in database.
My website is www.dingy.cz
For example word:
Zvířátka show fine in database but when it is push on website it shows only as Zv�?�tka.
I went true whole ...
0
votes
2
answers
49
views
SQL Server JDBC: Latin1 to UTF-8
There is a table in SQL Server with collation SQL_Latin1_General_CP1_CS_AS.
The table has a column varchar(35) with the same collation SQL_Latin1_General_CP1_CS_AS.
The column contains a string with ...
0
votes
1
answer
107
views
different utf8 encoding outputs between Python and Haskell
I'm trying to encode a has which is computed by hmac sha256 in Haskell and sometimes my function has different output when compared to it's Python counterpart.
This is the python funciton:
import ...
0
votes
1
answer
30
views
Cryptojs decryption not working for encrypted url param
When I am running the below code block it's working fine.
var request = "testing decryption";
var encryptedRequest = CryptoJS.AES.encrypt(request, 'somekey');
console.log(encryptedRequest)
...
0
votes
1
answer
20
views
C# HttpClient save response with MIME "text/plain" as an UTF-8 string
I'm sending a request with HttpClient to a remote endpoint. I want to download the content and save it to a file as an UTF-8 string.
If the server would respond with the proper Content-Type text/plain;...
-1
votes
0
answers
47
views
How do I read non UTF-8 from a pipe? [duplicate]
I'm writing a simple code that reads from pipe and prints the output. The code is the following (similar to this):
use std::io::{self, BufRead};
use atty::Stream;
fn load_stdin() {
if atty::is(...
1
vote
2
answers
54
views
What is the string representation of a bytes object really and how to get back the bytes object?
I have some text encoded to bytes using utf-8 encoding. When processing this text I incautiously used str() to make it a Unicode string because I assumed this would automatically decode the bytes ...
0
votes
1
answer
44
views
"utf-8' codec can't decode byte 0xdb in position 1: invalid continuation byte" - aws lambda
I am trying to decode a JSON formatted mqtt message that is sent to an aws lamda function
def lambda_handler(event, context):
eventText = event['data']
eventText = base64.b64decode(eventText)
...
0
votes
1
answer
31
views
Why does bitbucket display UTF8 characters as junk values while creating PR?
I am testing different languages in my project. I use IntelliJ for development and I have set both Global encoding and project encoding to UTF8 in IntelliJ settings. All the language files look normal ...
1
vote
0
answers
42
views
Can I use a std::fstream with UTF-16 encoding?
The question came from using std::wfstream which uses wchar_t as the character type, i.e. it is std::basic_fstream<wchar_t>.
However wchar_t is ...odd. Particularly on Windows it is 2 bytes. So ...
0
votes
0
answers
54
views
warning "using locale code page other than 65001 ("UTF-8") may cause problems" after R 4.2 update
I work with multiple locales/languages on Windows, and after my recent update to R 4.2, I keep seeing a warning " using locale code page other than 65001 ("UTF-8") may cause problems&...
0
votes
0
answers
15
views
PHP JSON UTF-8 encoding problem from prepared request with mysqli [duplicate]
I got this php function to check and return the results of a query :
function get_result($query, $types, $params) {
global $client;
$stmt = mysqli_prepare($client, $query);
...
-1
votes
0
answers
41
views
NetBeans insists on saving in wrong character set [closed]
I have an issue with NetBeans (12.0) saving a JSP file. The issue is the same as one addressed in a very old question here: Saving JSP as UTF-8 in NetBeans, but the remedies don't appear to work for ...
0
votes
1
answer
34
views
Storing polish character as variable in PHP
I am creating a simple registration form, in which you must type your nickname into textarea. The problem occurs when user uses polish characters (ś,ż,ć etc.). When I try to display the whole string ...
0
votes
0
answers
17
views
Emoji's not handled properly in Rstudio while displaying output in HTML
I am facing an issue while dealing with emojis on RStudio. I am using macOS and LibreOffice to open my CSV files in UTF-8 and it displays emojis perfectly. But when it comes to RStudio it cannot ...
0
votes
2
answers
36
views
How do I to export utf8mb4 mysql data using php to a csv file
I am looking for help on a csv file export.
I have a mysql database encoded as utf8mb4 (unicode_ci) with a table using collation utf8mb4_unicode_ci for my fields. The data contains special characters ...
0
votes
0
answers
23
views
PDF created with phantomjs seems to use ASCII
A am running an application in Meteor on Galaxy. I use phantomjs to generate a PDF-file from an HTML string. When I print the HTML-string in the console, all characters appear correct, but the ...
0
votes
0
answers
60
views
Iterator of `char32_t` from an UTF-8 file
Suppose I have an std::ifstream open on an UTF-8 encoded file. Is there a way to obtain an iterator over corresponding UTF-32 characters as char32_t?
Something like:
for(char32_t char : as_utf32(...
0
votes
0
answers
22
views
R eval parse encode utf-8 uses windows 1252
I have a R code file that is encoded with UTF-8. In want to use it on windows.
When I source the file (with encoding='UTF-8'); I get no error but when call the function within the file: I get an Error ...
0
votes
2
answers
62
views
'UTF-8' decoding error while using unireedsolomon package
I have been writing a code using the unireedsolomon package. The package adds parity bytes which are mostly extended ASCII characters. I am applying bit-level errors after converting the 'special ...
1
vote
1
answer
56
views
UnicodeEncodeError: 'ascii' codec can't encode character '\u2019' in position 546: ordinal not in range(128) only with lots of text [duplicate]
I am getting the above error when trying to run an automated email script. If I delete most of the body of the email it works fine. However, when I add over some number of characters (all are upper/...
0
votes
0
answers
20
views
HTMLAgilityPack Why are Attributes not correctly encoded after reading Utf-8 Encoded File
I am reading a HTML file that is utf-8 encoded:
htmlDocument = new HtmlDocument();
htmlDocument.Load( filePath );
The file contains a valid <meta charset="UTF-8"> tag, and after ...
0
votes
2
answers
67
views
postgres database issues with encoding UTF-8
I have created a docker image with this command docker compose up -d
where I was able to load pgAdmin instance in http://localhost:5050/browser/
create a database and table in the same , credentials ...
-1
votes
1
answer
55
views
Convert encoding of a csv file from utf-8 to shift-jis in python
I have a CSV file with utf-8 encoding. I want to change it's to shift-jis csv file using python code. Is it possible? How can i do it?
0
votes
1
answer
52
views
Umlauts and special characters preservation (Send-MailMessage)
I am having a problem with the formatting of an e-mail. I have special characters in the body and they are not being displayed correctly, I have tried several solutions but the e-mail keeps arriving ...
0
votes
0
answers
30
views
R markdown knitting error and R starting error after installing the packages extrafont, sysfonts and showtext / re-installation of R
In desperation I try to re-establish the ability for knitting markdown skripts, as this is no more possible after installing 3 packages concerning other fonts in graphs. This is what happened:
I ...