Questions tagged [character-encoding]
Character encoding refers to the way characters are represented as a series of bytes. Character encoding for the Web is defined in the Encoding Standard.
15,074
questions
0
votes
1
answer
29
views
The Java Eclipse Console dont show the correct French characteres
I’m using Eclipse IDE 2022-09 (4.25.0) and when I print a French text like Périneau, the console return P�rineau.
example:
String str = "Périneau";
System.out.println(str);
console:
P�...
-1
votes
0
answers
47
views
How to extract information from an on-line dictionary using web scraping [closed]
I'm doing some research on Medieval Catalan and I need to identify some unknown words that appear in the texts I'm analyzing. I know a little Python and I wanted to automatize what otherwise is a very ...
0
votes
0
answers
37
views
German Umlaute in Putty are not recognized as letters [closed]
The code
#!/bin/bash
read IN
echo $IN
LETTERS=$(echo $IN | grep -o -e "[[:alpha:]]" | tr -d "[:space:]")
echo $LETTERS
executed with a connection over Putty does not what I ...
-3
votes
0
answers
59
views
How to add @ symbol to string at beginning [closed]
How do I add the @ symbol to the beginning of a string so that the string is interpreting in the following way
string inputString = "\u0002;E\r\n";
string outputString = @&...
0
votes
0
answers
34
views
How to fix Turkish character encoding issue in XAMPP MySQL server?
Mysql Turkish Characters problem
Im using xampp mysql server for server.
But im trying to use it from the terminal/cmd.
When i query smth with turkish letters like "oğuz" , it returns "?...
0
votes
0
answers
15
views
How to connect to a database if the password or username given by the user in SQLAlchemy's connection string contains an @,: or / symbol [duplicate]
I am taking a connection string from user input and am getting an error if the given string contains @,/,:, etc. So, how can I resolve it?
Example:
from sqlalchemy import create_engine
...
-2
votes
1
answer
73
views
How to replace "/" with "\/" in python when writing to a file [duplicate]
I want to replace / with a \/ in a string. For an example
original_string="https://stackoverflow.com/questions/"
modified_string="https:\/\/stackoverflow.com\/questions\/"
...
0
votes
2
answers
50
views
Why is reader in java reading characters wrong?
So i have been trying to read different characters from an unknown file (my own file extension .xs) but it doesn't seem to work. For example it reads this character '¸' as 65533 and its ASCII code is ...
0
votes
0
answers
43
views
Problem about read thai language text file and Insert text thai language in to database [duplicate]
Use Javascript for coding
Thai language text file is enconding with ANSI
setting Collation in SQL is utf8_unicode_ci
show text before insert to database it can be show thai language like this ...
0
votes
1
answer
50
views
Why do Python and Java behave differently when decoding GBK
This question is related to How to decode bytes in GB18030 correctly.
I would like to decode an array of bytes which are encoded with GBK, but found that Python and Java behave differently sometimes.
...
0
votes
0
answers
52
views
How to decode bytes in GB18030 correctly
Recently, I am studying GB 18030, and found that some characters cannot encoded/decoded correctly when they are mapped to the private user area (PUA). For example, according to the specification of GB ...
0
votes
0
answers
56
views
Printing special characters with umlauts in Scala
i am having trouble printing special characters with umlauts.
I have a Scala program with print("äö") and it's printing ??.
I have my VSCode set to UTF-8, I have specified scala to use UTF-8 ...
0
votes
2
answers
62
views
How to decode `piexif` metadata strings containing both plain text and bytes characters?
I have a string containing both human-readable parts and bytes/control characters (I guess), like the following:
weird_string = 'Nikon\x00\x02\x00\x00\x00II*\x00\x08\x00\x00\x00\x15\x00\x01\x00\x07\...
0
votes
0
answers
27
views
Dart - Dio - Converting Latin2 text to UTF-8
How is it possible converting to UTF-8 from Latin2 by using dio library?
I fetch a website content by using the dio
final response = await dio.get(url, options: Options(
responseType: ...
0
votes
1
answer
55
views
i need some basic godot enemy movement script in godot 4.0 or a way to make the enemy movement in godot 4.0
All I need is an enemy movement script or some instructions on how to make it.
Your efforts to help me will be greatly appreciated.
Personal Experience: I'm kind of still new but are gaining knowledge ...
0
votes
1
answer
39
views
During Outlook item registration Greeks and non Latin characters seems being recognized and replaced by ??? MailItem.SaveAs(path, [Object Type])
Dears,
By using * MailItem.SaveAs(sing path, [Object Type]) *
during Outlook item (.msg) registration on temporary folder (file system) to be processed after,
all Greek and non Latin characters seems ...
0
votes
0
answers
32
views
PHP Exec Shell Returns ASCII UTF8 Needed/Intended
Short version:
/usr/local/bin/aws s3 ls s3://bucketlocation/folder --recursive from shell gives
file - 复件(1).tiff
where as from PHP (shell_exec() or exec() same result) I receive:
file - ??(1).tiff
...
2
votes
1
answer
75
views
String.getBytes("8bit encoding") returns 2 bytes per symbol
I'm trying to understand character encoding for Strings in Java. I'm working on Windows 10 and the default character encoding is windows-1251. it is 8-bit encoding character. So it must be 1 byte for ...
0
votes
1
answer
75
views
GET request with emojis on url fails: 'utf-8' codec can't decode bytes
I am trying to make a request using requests module in python.
import requests
url = "https://www.foo.bar/topic/512174-some-text-�-text-some-�☀️/"
r=requests.get(url)
I have tried both ...
1
vote
1
answer
155
views
Projecting number of bytes for double, triple etc encoding cp1252->UTF-8
So I recently came across this question:
How do I recreate these special characters when testing a website?
where the user describes a situation in which a client sends a string with accented ...
1
vote
1
answer
53
views
Spring MVC with Tomcat UTF-8 encoding
At the moment I'm trying to get started with Spring MVC.
Something wrong with my encoding.
Every time response header is:
Content-Type: text/html;charset=ISO-8859-1, so all non-ASCII characters appear ...
3
votes
1
answer
32
views
Why is there no $encoding parameter in htmlspecialchars_decode()?
If we check out the documentation of the htmlspecialchars() function in PHP, we see that it has an $encoding parameter to specify the encoding of the input string.
Now, conversely, I expect the ...
0
votes
1
answer
35
views
psycopg2 set proper encoding
I am using psycopg2 as database adapter.
The connection looks like
conn = psycopg2.connect(host=hostname,
user=username,
password=password,
dbname=database
)
In the result of my query i have ...
0
votes
2
answers
64
views
Hebrew character looks like a division sign when using iconv with FPDF
When I use iconv to convert encoding from UTF-8 to windows-1255, all the Hebrew letters show up correctly, except for ק, which shows up as the division symbol: ÷:
$pdf->AddFont('Arial', '', 'arial....
0
votes
0
answers
27
views
Using Turkish letters in RStudio
When I save an R script containing Turkish letters such as ş,İ,ı in RStudio with ISO-8859-1, and then open up the same R script again the Turkish letters turn into s,I,i. How can I avoid this ...
-2
votes
1
answer
71
views
Korean currency symbol converting to "?" while writing to CSV file using Java
I am exporting a table into CSV using Java code. The code can have price in any currency.
Following code is used to do that
If Currency is GBP is Euro
StreamUtils.copy(writer.toString(), Charset....
0
votes
0
answers
80
views
Rewrite url in reverse proxy in golang
i have a problem with send a special char in url by reverse proxy in golang. This is my function:
func TurnOnDevice(w http.ResponseWriter, r *http.Request) {
API_TURN_ON := "/relay/0?...
0
votes
0
answers
45
views
How to print "\xc9" properly to terminal in Windows?
I have trouble printing some special characters to the terminal using C. But I can do it with python. I think it has something to do with the encoding, like UTF-8.
ref: https://www.codetable.net/hex/...
0
votes
0
answers
30
views
Can windows console application work with mixed charset?
I made a C program that had the following requirements for input string or it's substring/chars:
Must be able to user input from console, and/or read input from text file
Must support reading, ...
0
votes
0
answers
30
views
Google Cloud function can't display non-ascii character
Have a encoding problem for java cloud function in GCP, non ascii character can not be displayed, e.g.
public void accept(PubSubMessage message, Context context) throws Exception {
String text = "...
0
votes
0
answers
26
views
Getting codepage / encoding for windows executables called with subprocess.check_output from python
I have a similar issue to How to get the output of subprocess.check_output() python module? but the solution there does not work for German Windows.
I execute the following script in python 3.10 in ...
1
vote
1
answer
88
views
How to add CP932 charset to iconv in Alpine
I want to convert CP932 string to UTF-8.
In Ubuntu I can convert string by a command
iconv -f CP932 -t UTF-8 [input_file]
But the iconv in Apline 3.17 fails with a error: iconv: source charset CP932: ...
0
votes
0
answers
38
views
Maven install has wrong platform encoding
Whenever i do a mvn -version i get the following response
Apache Maven 3.9.1 (2e178502fcdbffc201671fb2537d0cb4b4cc58f8)
Maven home: /Users/xxxxx/apache-maven-3.9.1
Java version: 11.0.18, vendor: ...
0
votes
1
answer
41
views
unmappable character (0xC2) for encoding US-ASCII
I am practicing on of methods of ArrayList set().I got very crazy result here:
How to escape this error and successfully to set new value for ArrayList?
I expected this output : [Forever Garcinia ...
0
votes
1
answer
38
views
Same text in UTF-8 but different in ANSI
I have a Notepad++. The Encoding is UTF-8, in notepad I have two text
Thành
Thành
But when i use Find dialog to search "Thành" the result has only 1 result.
I change the Notepad++ ...
0
votes
0
answers
31
views
python3 - Flask - Issue printing accents - encoding problem
I have a Flask app, in Python3.10, running in "prod" via Apache2.
My app is not able to handle and print accents at all, and this is driving me crazy! :|
What i need, is my app being able to ...
0
votes
0
answers
40
views
MimeMessage.writeTo(outputStream) ignores the encoding on the html content of the mail. It's replacing the apostrophes (') with (?).How to avoid this?
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
mimeMessage.writeTo(outputStream);
That's the code where I write the entire email Mime content (including the headers) to an output ...
0
votes
2
answers
76
views
How to save thorn with ansi encoding in PHP?
I have to save a text file with ansi encoding, containing the special character "thorn" or "þ" in PHP. When I simply place the thorn in PHP, it is going to look like "ţ" ...
0
votes
1
answer
297
views
maven 3.9.1 failing to build project with error: `filtering {file} to {file} failed with MalformedInputException: Input length = 1 -> [Help 1]`
I'm trying to build an existing Java project on my local machine, but it's not working since I upgraded Maven on this machine from 3.8.x to 3.9.1 yesterday. The error message I get is:
[ERROR] Failed ...
2
votes
2
answers
54
views
Which XML encoding for storing filenames?
I would like to store raw filenames in an XML document but the encoding doesn't allow it.
Here's how you can generate a test.xml with the shell:
#!/bin/sh
cat <<EOF > test.xml
<?xml ...
0
votes
1
answer
90
views
Parsing HTTP body in Java
What I am trying to achieve:
I am currently developing a proxy where the goal is to alter the body received in the server's response. (The proxy shall only support HTTP, not HTTPS).
As an example of ...
0
votes
0
answers
43
views
Issues with Mysql JSON Datatype for UTF-8 encoding JAVA Spring
The database collation is utf8_general_ci and the table encoding to utf8.
But couldnt add encoding to the JSON column. Thats odd
Java Connection with Mysql has The connection string specifies the ...
1
vote
1
answer
99
views
How exactly is the length of a Postgres VARCHAR determined (UTF-8)?
The PostgreSQL documentation is very specific that VARCHAR(n) refers to a type which can store strings up to n characters (not bytes) in length. What a "character" actually is depends on the ...
1
vote
0
answers
29
views
MSTest unit test on .NET Core only provides seven encodings. How can I reach the ANSI one? [duplicate]
I'm unit testing a .NET Core solution in Visual Studio 2022 on Windows.
In my solution I want to output an ANSI string stored in a byte buffer, but that doesn't seem to work.
This is an extract of the ...
1
vote
1
answer
111
views
How to correctly use codecvt_byname (C++17) to encode latin1, and then UTF-8 for use in JSON
I am (desperately) trying to prepare a byte array (copied from a PLC, where they construct the "string" as a byte array, locale/encoding is German, French, etc) for use in nlohmann::json, ...
0
votes
2
answers
76
views
What is the encoding of this data? Is it corrupted?
I have this file which store some strings data for i18n. Here is the raw data in HEX:
70617573652E70726F636573732E6D73
673D20496D706F737369626C65206427
696E746572726F6D707265206C652070
...
0
votes
0
answers
42
views
How to preserve string literal encoding between different systems?
I'm writing code on a windows machine, but when I run it on Linux, non-English characters are printed incorrectly to stdout.
The goal is to print the character Ä.
Source code:
import sys
import locale
...
0
votes
0
answers
38
views
Where can I find GCC or Clang's documentation about how they process ordinary character encoding?
[lex.charset] p9 says
The ordinary and wide literal encodings are otherwise implementation-defined.
char c = '@';
I wonder how GCC or Clang defines the ordinary literal encoding and how the ...
0
votes
1
answer
32
views
Is it possible to create your own character symbols?
So I have tried using private character editor for creating character symbols, but it is impossible to export them to be shared with other users. However I was wondering is there a way to create your ...
1
vote
1
answer
64
views
Wrong encoding / chars in email via lettre
I'm sending email from a rust program.
Email text is in a Postgresql database with encoding UTF8. I query database via postgres crate: https://crates.io/crates/postgres
I'm working on Windows. If I ...