Questions tagged [utf-8]
UTF-8 is a character encoding that describes each Unicode code point using a byte sequence of one to four bytes. It is backwards-compatible with ASCII while still supporting representation of all Unicode code points.
22,237
questions
0
votes
1
answer
43
views
Count words in text
I need to count the number of words and the number of unique words from a UTF-8 text file, the text can contain any language. I tried the next option, but I get more words than in Word or online ...
-2
votes
0
answers
30
views
Co-developing a python project on Windows and macOS encoding issues [closed]
I am developing an application using pycharm on mac default encoding utf-8. When I open the same project in pycharm on Windows, I get scribbles.
((
I set the encoding utf-8 in pycharm - no changes. ...
-1
votes
0
answers
26
views
Scripting.FileSystemObject::OpenTextFile() destroys UTF-8 BOM file header on Asian Operating Systems [duplicate]
I try to change the file content of a file and like to save the result in a new file, by copying/changing the file line by line.
This is a test example in vbScript which shows the problem when reading ...
0
votes
0
answers
8
views
R - Exporting plot - gsub() issue
I have a problem with exporting/saving plots in R. It is quite interesting because it started after I downloaded R 4.3.2 (Windows).
numbers <- c(20, 80, 110, 70, 98, 100, 215, 65)
plot(numbers)
...
0
votes
0
answers
39
views
Saving utf-8 text as ansi file in python
with open("C:/temp/Python/infile.csv", "r", encoding="utf-8") as csv:
for line in csv:
line = line.strip()
cells = line.split(";")
...
-1
votes
0
answers
19
views
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc1 in position 32: invalid start byte [duplicate]
I run display.py, it had an error like this. Can you teach me how to fix this? thanks
error
I have tried searching for solution but almost them is for csv file, I don't know what the reason is. Please ...
0
votes
1
answer
35
views
postgres COPY FROM STDIN WITH encoding error with UTF8 by using libpq
I created database by following command:
CREATE DATABASE myendb
ENCODING 'UTF-8'
LC_COLLATE 'en_US.UTF-8'
LC_CTYPE 'en_US.UTF-8'
TEMPLATE template0;
I tried to insert Cyrillic symbols encoding by ...
-4
votes
0
answers
65
views
are there any characters that std::cin cannot read? [closed]
I need to crash my programm work. Is there any symbol that std::cin can't handle properly and will throw an error?
i tried to generate some invalid utf-8 symbols using python:
invalid_utf8_sequences = ...
0
votes
0
answers
43
views
How to show `\` instead of special character `₩` in path separator for Windows Forms text-box?
My Windows Forms app displays a file path in a text-box. For our customers in Korea, this is shown as C:₩Users₩abc123₩my₩path instead of C:\Users\abc123\my\path. However, their command prompts, ...
0
votes
1
answer
116
views
Handling UnicodeDecodeError when reading a file in Python
I'm working on a Python project where I need to read data from a file. However, I'm running into issues with UnicodeDecodeError. I've looked into similar questions on Stack Overflow, but I couldn't ...
-1
votes
0
answers
22
views
Mysql collation diffetence between utf8mb4_unicode_520_ci and utf8mb4_bin? [duplicate]
In a MySQL table where I need to store many websites titles/description/kewyords (so "very" multilanguage), will I be able to store the same chars if I use utf8mb4_bin collation instead of ...
0
votes
1
answer
109
views
Decode UTF-8 escaped string
I have searched all around the internet for a solution for my specific issue, but to no avail.
I'm trying to decode the following string:
\x3cdiv\x3e\x3cdiv style\x3d\x22padding-bottom:12px;padding-...
1
vote
1
answer
40
views
How to detect in PHP that pensé is the same as pense [duplicate]
How can I do a test in PHP to compare two words (one with accent) and detect that is the same word ?
e.g pensé vs pense
Basically I told it to load certain words from MySQL table, and with MySQL ...
-2
votes
1
answer
61
views
Unable to Output Chinese Characters in CLion Using MSYS2 MinGW64 Despite UTF-8 Settings
I am developing a C++ application in CLion using MSYS2's MinGW64 compiler. My project involves outputting Chinese characters, but despite setting everything to UTF-8 (Global/File/Terminal encodings in ...
0
votes
1
answer
126
views
Converting a byte array to a string and converting it back to byte array in same encoding producing different results
I'm generating a random 20 bytes long array and want to convert it to a string, in order to use it as a random token for an API call.
However when I convert it back to a byte array (just for testing), ...
0
votes
0
answers
39
views
Psql used with Azure runbook can't handle UTF-8 encoding
The setup is like this, we have an Azure runbook which is using a hybrid worker with psql to insert data in to a CosmosDB running PostgreSQL.
The issue is that the data contains Swedish characters ...
1
vote
0
answers
32
views
How do I resolve UTF-8 encoding error with read_csv? [duplicate]
I am doing an assignment for my data analytics class and when I attempt to read the csv file for the "main_genre_df" it is giving me an encoding error? I have looked online and it told me to ...
0
votes
2
answers
86
views
utf8_encode and mb_convert_encoding Conflicting Resuts [duplicate]
I have a very annoying issue when transferring data from PHP (which is set up with default_charset = 'UTF-8') to MySQL, on an IIS environment with web.config set to:
<globalization requestEncoding=&...
1
vote
1
answer
80
views
How do I get UTF-8 to work flawlessly in modern PowerShell on Windows?
I have a C++ program which outputs raw UTF-8 and works flawlessly on Linux, but on Windows shells the output is not as nice. "®" turns into "┬«", "©" turns into "┬⌐&...
0
votes
0
answers
12
views
C# Razor Loading a css stylesheet form the server and embedding it in the view
I would like to bring an external stylesheet into my web application's _layout.cshtml view.
I have so far been able to read it from wwwroot into a variable but I have hit an encoding issue.
My basic ...
0
votes
1
answer
73
views
Creating a file from a absolute path, which is containing a german umlaut
I have a problem with creating a file from a absolute path, which is containing a german umlaut. (Working on windows)
This is a example code:
public void createFile(String path) {
System.out....
-1
votes
0
answers
14
views
UTF-8 Encoding of Chinese Characters - php displaying string correctly, but not individual characters [duplicate]
I'm trying to get one chinese character at a time from the line 道可道,非常道。
echo "道可道,非常道。";
echo substr("道可道,非常道。",0,1);
The first line works fine, and outputs the ...
0
votes
1
answer
81
views
Encoded image sent over redis from Cpp app to Rust app, Invalid UTF-8- TypeError
I'm sending an encoded cv image over redis pubsub from Cpp app:
cv::Mat output_image;
output_image = cv::Mat(pic_height, pic_width * 2, CV_8UC3);
cv::imwrite(filename, output_image);
std::vector<...
0
votes
0
answers
16
views
Can SparkJava encoding all request data as UTF-8 by default?
I wonder if there is a way to configure SparkJava to encode all request data with a character set of my choice regardless of it comes from query parameters or POST data. I have a number of web app's ...
0
votes
0
answers
33
views
Excel recognizes as correct an utf-8 encoded file (in Import Data) but failed to open it correctly
I'm trying to open an utf-8 encoded csv file in Excel for Windows 10.
When opening it directly, characters appear wrong. If I go to the Data menu >>Import Data, a window with the dataset appears,...
3
votes
1
answer
103
views
How can I write utf-8 characters to a csv file with C?
How can I write UTF-8 characters to a CSV file in C? I want to write letters such as Ş, Ğ or İ. Also I cannot write in different columns. There is something wrong with my PC, I guess: it does write ...
0
votes
0
answers
16
views
Kivy large text inside lable reversed and inserted from buttom to top
I have a program in kivy and I want to insert code text from database to label. Although I checked in static text but same result.
The problem is the large text inserted from bottom to top, and we ...
0
votes
1
answer
57
views
Pandas on Linux importing UTF 8 (BOM) csv with BOM header as cleartext in column names
Difference on importing csv data on Linux and MacOS
Hello everyone,
when importing a csv file with pandas.read_csv using UTF8 (BOM) encoding under Linux, the first column name contains the BOM as ...
0
votes
0
answers
17
views
github showing extra characters for my properties file
Why does my Github desktop shows data like this? same is working fine in eclipse.
-1
votes
0
answers
31
views
Haskell read and parse utf8 chinese file by parsec encoding question [duplicate]
I am reading and parsing a Chinese txt file.
Only reading and writing without parsing can output Chinese.
After parsing, Chinese would turn to someting like \12304\12304CB8\12305\24615\33021\20248
...
0
votes
1
answer
38
views
prevent Jsoup from converting utf-8 to utf-16
I would like the following to return the encoded bullet point the same way it was supplied:
Jsoup.parse("<div>•</div>").text()
but instead I get a utf-16 string out ...
0
votes
0
answers
24
views
Partial display of text
Cyrillic text is partially displayed
this is how looks
html was:
META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1
I change in:
META HTTP-EQUIV="Content-Type&...
-2
votes
2
answers
78
views
Storing null bytes in Javascript string literals
Consider the following HTML:
<!DOCTYPE html>
<html>
<body>
<script>
const a = " ... ";
for (let i = 0; i < a.length; ++i) {
...
0
votes
2
answers
75
views
How can I insert long string into UTF8-collated varchar(N) column?
I have data in VARCHAR(MAX) source, that I need to insert into another table as VARCHAR(N).
The collation I need to use is UTF8 (specifically Czech_100_CI_AS_SC_UTF8).
The problem is, even if I cut ...
0
votes
0
answers
21
views
Issue with Kannada Language Rendering in WordPress Tags and URLs
I hope this email finds you well. I am writing to seek assistance with a Kannada language rendering issue I am facing on my WordPress website.
I have been attempting to use Kannada characters in my ...
1
vote
1
answer
75
views
How to diagnose character encoding issue
I am having trouble identifying an issue that seems to be related to strange characters that exist in a Postgres database. I'm pulling data out of Postgres using Java and loading it into BigQuery. ...
0
votes
0
answers
23
views
Pushing picture to gitlab with python-gitlab
I am trying to push a jpg picture to a gitlab repo with python-gitlab. My code looks like
gl = gitlab.Gitlab(GITLAB_URL, PRIVATE_TOKEN)
project = gl.projects.get(PROJECT_ID)
with open(file_path, 'rb')...
1
vote
1
answer
89
views
A weird encoding problem in comments of Batch scripts
I added an announcement of AGPL-3.0 licence in the header of my Batch script (saved in UTF-8). In addition, for the convenience of Chinese developers, I also added a Chinese translation link below it, ...
0
votes
0
answers
25
views
Text widget in Tkinter not accepting Pashto writing that is a UTF-8 encoded language
Pashto is a UTF-8 language, I am working with Tkinter library of python and make a Text box to write in it utf-8 languages like Pashto, but when I am writing in it, it does not work just show me ...
0
votes
0
answers
34
views
add lua-utf8 support from luarocks
I am on ubuntu 22.04 desktop
i installed luarocks, luarocks install luautf8, i added this on my code local utf8 = require 'lua-utf8' and while i am running it on my studio code i get the following
...
1
vote
1
answer
50
views
Decode byte string to Cyrillic in Python
I have a byte string like this, it should be Сравнение in Cyrillic characters:
a = b'Сравнение'
Decoding it ...
1
vote
1
answer
69
views
How to handle utf-8 non-breaking spaces in CSV
In a Python script I am using openpyxl to read an XLSX file. I move some metadata around, and ultimately writing out a new CSV. When I open that CSV in Microsoft Excel, there are whitespaces in a ...
0
votes
0
answers
24
views
ASP.Net return XML document from XNode with proper encoding [duplicate]
I want to return an XML document represented as an XNode from asp.net
I have
public ContentResult SaveXml(XDocument doc, int statusCode){
StringBuilder b = new();
using var writer = new ...
0
votes
0
answers
38
views
Python - textual - question marks instead of borders in dos prompt [duplicate]
I am running my python code in vs code and it displays fine
but in a windows command line I get question marks where borders should display
I use a tutorial code:
class Hello(Static):
BORDER_TITLE ...
0
votes
0
answers
45
views
UTF8 error after pg_escape - Postgresql and PHP
I have the following piece of code that reads a txt file and imports it to a postgresql db. I used pg_escape_string for all the text fields that could possibly contain special characters. Then I ...
2
votes
2
answers
61
views
UTF-8 input and using XGetICValues
I have found an example (accepted answer) of UTF-8 input but I wonder if that example is correct. The result of XGetICValues may be passed to XSelectInput:
unsigned long mask = 0;
if (!XGetICValues(...
0
votes
1
answer
33
views
UTF-8, Unicode in SML/NJ
I am playing with SML/NJ (version 110.99.4) on Windows 10.
I have a structure containing a text file in UTF-8 encoding:
...
let
val s:string = "søk"
in
print s
end;
.....
2
votes
4
answers
170
views
How to handle values greater than 255 as returned by toupper() in recent macOS with UTF-8 locales
The problem, the code below is trying to address, is on how to effectively detect that an UTF-8 based locale might be in use so that all code points above 127 are not queried for ctype attributes, as ...
0
votes
0
answers
43
views
How to quantify number of errors due to character encoding?
I have a data process where strings are passed through many applications/databases with different character encodings. Some of the steps along the way are UTF-8 some aren't but the end database where ...
0
votes
1
answer
50
views
Can't display Chinese characters in R plot
Chinese characters don't display in a basic R plot:
plot(c(1,2), c(1,3), xlab = "基本", ylab = "收入")
I tried adding family = 'Arial Unicode MS' to plot, and I tried setting Sys....