Questions tagged [awk]
AWK is an interpreted programming language designed for text processing and typically used as a data extraction and reporting tool. AWK is used largely with Unix systems.
32,513
questions
-1
votes
0
answers
16
views
Unbelievable: Awk rounding a float to integer got weird result: 123.12*100 -> 12312(good) but 1234.12*100 -> 123411(bad)
awk '{printf "%d\n", (1234.12*100)}' <<<""
outputs
123411
It is supposed to be 123412.
Awk version: GNU Awk 4.1.4, API: 1.1 (GNU MPFR 4.0.1, GNU MP 6.1.2)
What am I ...
0
votes
0
answers
11
views
How i can delete duplicate lines but leave the line where field have particular value
I need to delete duplicate lines in a pipe separated text file and leave the line where field 11 and 16 contain the value 9.
how can I do? for example if I have 3 records
L|20230502|0390286|F|0|0000|...
-2
votes
0
answers
40
views
awk command: how to delete lines from file with "only blank characters" or only <tab> or <cr>? [closed]
I made an awk script to clean up commented files (gforth in emacs), See the code below. However, few lines with following characters are still in the output file (should be gone):
(backslash) then (...
1
vote
4
answers
64
views
Is there a better way to parse the output of a command that produces a string with multiple columns than using awk '{print substr($0, 68)}'? [closed]
I run a CLI command that outputs various database values. One CLI command example is:
show attributes attr=0x1006e mh=0x12003ea | awk '{print substr($0, 68)}' | grep -v Value
The output looks like ...
0
votes
2
answers
74
views
How to fill the values to a shell script from tsv file and create new file from 1st column value
I have a data.tsv file with 100 entries, as below
ColumnA
ColumnB
ColumnC
ColumnD
ColumnE
Cell 31
Cell 4
Cell 3
Cell 5
Cell 8
Cell 21
Cell 2
Cell 5
Cell 6
Cell 9
and I have a template.in script ...
0
votes
2
answers
66
views
Find nearest range higher and lower inside another range
I have rewritten this question many times, as while writing it out, I have come up with some ideas and have managed to get a very slow, but working solution. It seems writing out what one needs to do ...
-2
votes
0
answers
32
views
Append multiline from a file using AWK after a multiline patter search [closed]
I have a file which is of 400 Lines. I need to search multiple lines which is a pattern and then after the pattern it needs to append few more lines from
a file. The command i am using is
awk -v RS='^$...
4
votes
8
answers
91
views
Shell command to add double quotes for number surrounded by dots in property keys
I have following property keys
connection.party= 0.0.0.0
remote.app.328.port= 5432
remote.server.url="https://someurl.com?q=78.78"
remote.conncetion.7="basic"
remote.52.base= "...
-1
votes
4
answers
71
views
Calculating the median value from a TSV file for each month [closed]
I have a TSV file and I am trying to perform some analysis. I have a month column and the data is numbers 1-12 which corresponds to the month (Jan=1, Feb=2 etc). I am trying to setup a counter so that ...
0
votes
0
answers
63
views
How do you issue a 'break' command in a nested while/do/done containing multiple loops?
#!/bin/sh
set -x
#########################################################################
# This script will export a listing of all devices and where each device
# lives (...
-1
votes
0
answers
25
views
awk single regular expression in matching two different patterns [duplicate]
My awk version is GNU Awk 5.0.1, API: 2.0 (GNU MPFR 4.0.2, GNU MP 6.2.0).
Both
echo '{{' | awk '/{{2}/ { print "foo" }'
and
echo '{{2}' | awk '/{{2}/ { print "foo" }'
are ...
2
votes
2
answers
63
views
awk not finding a string value with special characters in a csv file
I'm just trying to find a way to check if a string value exists in a csv column using awk command. It works fine except for the strings which contains special characters like ( or [.
It worked for ...
-1
votes
0
answers
41
views
BASH Append a backslash before a space in a string and show the backslash with the space [closed]
Good Day
A file has two spaces in its name.
/storage/vbk-2023/2023-01-voi-decrypt/DAV202/2023-01-voi-decrypt - DAV202.vbm
I am trying to put two backslash at the two positions, as below, ...
2
votes
1
answer
66
views
Matching a regex metacharacter literally [closed]
My understanding of Regex's in AWK is that in order to match a Regex metacharacter literally (For example: +,$,^,*,etc) You must escape them, like so:
awk -F '\\+' 'program here'
However I've noticed ...
3
votes
5
answers
114
views
How do I remove the last 2 columns from a tsv file?
I am coding using bash on terminal through a docker container on my mac. I am struggling to figure out how to remove the last 2 columns on my TSV file. It has 7 total and the last 2 are not needed for ...
0
votes
0
answers
26
views
awk - turning date data into schedule [closed]
I have the following data:
Body,23/04/23,14/05/23,,,,,,
Tony,23/04/23,14/05/23,04/06/23,25/06/23,,,,
Sat,25/04/23,16/05/23,06/06/23,27/06/23,,,,
Gil,25/04/23,16/05/23,06/06/23,,,,,
Chan,25/04/23,13/05/...
-2
votes
1
answer
54
views
How to inject data through awk from CSV to Redis [closed]
I'm trying to inject data from csv file to Redis.
If I run the script in the terminal, I get a JSON format, but I do not know how to insert this data into Redis. I try using SET at the beginning and | ...
5
votes
5
answers
88
views
Extract 2 different and non-consecutive lines from text without creating temporary text files
I'm a musician and do a little programming on the side, so my abilities are limited, please forgive me.
Sometimes I need the LUFS level of multiple audio files and the quickest way (for me...) was to ...
0
votes
1
answer
35
views
chnaging a specific column in a specific row with for loop
I have a file with following content
### beginning of executable commands
### load the necessary module files
module load GCC/11.3.0 OpenMPI/4.1.4 CP2K/9.1
### your program goes here (a.out is an ...
2
votes
3
answers
65
views
How extract strings in first and last set of parenthasis?
I have a file: queueN.txt, that looks like this:
uid=524357(aali95) gid=524357(aali95) groups=524357(aali95),611684(com_perkinsd),695433(bios420)
uid=524534(aalman4) gid=524534(aalman4) groups=524534(...
1
vote
2
answers
77
views
AWK how to permanently remove a date field from each line of a datafile when date value is older than x days?
I have a todo list in todo.txt format.
I'd like to find the starting time ('t:2023-05-18') for all tasks with a date older than 10 days and remove the field from each matching line.
Sample Input Data
(...
1
vote
0
answers
57
views
using awk/sed to format log output into required CSV format [closed]
Hopefully this makes more sense, What I'm trying to achieve is two things, firstly to format the data in CSV format and secondly to perform a match on the node id to return if its patched as YES/NO ...
0
votes
2
answers
59
views
replace text in html with sed or awk
Could you please advice how to replace text in the html tags?
<link href="*" rel="icon" type="image/x-icon" />
I want to get instead of example above:
<link ...
3
votes
1
answer
48
views
Grab Nth occurrance of each element with AWK
This is the file:
AAACGCTGTGTCATTG-1-pere,1,2
AAACGCTTTGTCATTG-1-pere,3,6
AAACGCTATGTCATTG-1-pere,3,4
AAACGCTCTGTCATTG-1-mele,2,1
AAACGCTFTGTCATTG-1-pere,5,8
AAACGCTHTGTCATTG-1-mele,5,3
...
3
votes
3
answers
66
views
if else command in IBM AIX OS
I would check in the /etc/security/passwd file if a password is assigned to root user.
We have to types of results:
root:
password = *
or
root:
password = {ssha512}***********
i wrote ...
-1
votes
2
answers
88
views
Parsing a nested cell in a CSV file using the bash terminal or R
Attempting to create a scaled-down approach to reading in a LARGE (>150GB) CSV file into an R script by chunking it into smaller bits which can be read in sequentially.
Problem is, one of the ...
0
votes
1
answer
79
views
In Hadoop HDFS, delete several files older than x days and with spaces in the name (Not like UNIX)
I have hundreds of thousands of files in a hadoop directory and I need to debug them. I'm looking to delete files that are more than 3 months old and I'm trying to delete in batches of a thousand ...
-2
votes
1
answer
25
views
awk printf printing extra commas [closed]
My script, part of creating html tables in email using awk, printing additional commas (,)
`if (m_premium > mp) {printf ("%s,%.2f,%s\n","<td bgcolor=pink>",m_premium,"...
1
vote
7
answers
78
views
If the current line has the search string, do not print it and its previous line
How to do this?
If the current line has the string "9999-01-31", do not print it and its previous line.
Otherwise print.
e.g.
in.txt:
Test1 end1
Date1 : 2023-01-12 Date2 : 2023-07-23
Start ...
-1
votes
1
answer
33
views
Delete columns, not just empty them [duplicate]
I'm trying to delete the last 2 columns of a csv file with awk. I have tried using:
x=$(awk -v FPAT='([^,]*)|(\"[^\"]+\")' -v OFS=',' '{$10=$11=""; print $0}' ${filename} > ...
0
votes
2
answers
55
views
looking to convert first name last name to first initial last name from a getent output
I have to convert this.
getent passwd euser5
euser5:x:12345:20:Ester Jones:/home/euser5:/bin/csh
so far i have this.
getent passwd euser5|cut -d: -f1,5 |sed 's/:/,/'|tr '[:upper:]' '[:lower:]'
euser5,...
-2
votes
0
answers
21
views
How to print increment number when we use print command in xterm [closed]
i have to use below command
and i need to add increment number in name sonu
can anyone please help.
not able to get the
0
votes
1
answer
34
views
Using awk to find/replace the string in text file from defined variable
I have a input text file as below.
...
...
DataColumn No X Y W H Kind Ch Count1 Type Count2;
DataList 10
1 -45268.1732 50408.958800 0.0900 0.8998 0000000000000600 00800000 2 6 ...
-3
votes
1
answer
49
views
keep N number of duplicate rows (based on column 1) [closed]
I have a csv with this first column of indexes:
TTTGACTCATCCGATA-1-H0025_apex
TTTGATCTCAAGGCTT-1-H0025_apex
TTTGGAGAGTTTGAGA-1-H0025_apex
TTTGGAGGTAAGATAC-1-H0025_apex
TTTGGTTCAAGTGACG-1-H0025_apex
...
-2
votes
1
answer
60
views
AWK to handle bed files
I would like to grep and separate fields from bed files to generate a new bed file with these new arranged data.
I would go from here:
1 15903 rs557514207 G G,A RS=557514207;RSPOS=15903;...
2
votes
1
answer
59
views
Count columns from sacct output with awk in bash
I work on a slurm cluster and I want to find how much cpu and memory I am currently using. My main problem is distinguishing between 8G and 8Gc for memory usage. (the former is the raw required memory,...
0
votes
4
answers
83
views
Word count command returning incorrect row count for CSV files
I am running the following command to count the number of rows in a CSV file:
wc -l filename.csv
Sometimes the row count is correct and other times it is 1 less than the actual count. All of the ...
2
votes
2
answers
67
views
How can I make awk output more readable?
With this code I get the students with the highest and lowest value.
But I can't find how to turn it into a more readable output.
Code
awk -F ';' 'NR==1 { min = max = $2; mina = maxa = $1 " "...
0
votes
1
answer
51
views
How to recursively replace a string in all files of a folder with the contents of specific file outside the folder using bash script?
Problem:
I have a folder (Essays) containing many nested sub-directories and ".txt" files.
Only some files within this folder contain the string '[[RepeatedPara]]' somewhere among the ...
0
votes
3
answers
87
views
Delete lines in blocklist file, where the end of those lines match an entry from an allowlist file (Using Dnsmasq syntax)
This is a modification of a question I've previously asked, this time to account for Dnsmasq syntax in the blocklist. I'm trying to delete lines in an blocklist file, but only if the end of the ...
1
vote
2
answers
85
views
How to get max and min values in linux shell with multiple possibilities?
This shell script gets me the max and min values, but I would like to get those values and the associated students when there are repeats.
Code:
#!/bin/sh
Notamax=0
Notamin=0
alumneMax=''
alumneMin=''
...
0
votes
2
answers
71
views
How can I update an Awk command to work with a newer version of MacOS? [closed]
I am using the following Awk command which is based on this Stack Exchange post:
tail -n +2 *.csv | sort -t',' -k2 | awk -F',' '$2~/^[[:space:]]*$/{next} {sub(/\r$/,"")} $2!=prev{close(out); ...
0
votes
3
answers
86
views
output of awk if statement is wrong
I have a file with thousand's of rows and several columns like the next one:
38.13,38.05,2003-04-13,2.6,12,T22:47:52,Md
and I want to change the value of column 4 according to the value of column 7 ...
2
votes
1
answer
51
views
Need to get file name, line number, match position of a pattern in a file [duplicate]
I have a requirement where I have to get the file name, line number, match position (byte offset) of a pattern in a file.
I am using the grep command for this and so far am able to get the file name ...
0
votes
4
answers
94
views
Combine grep and sed to extract lines of data from a text file
I must extract data from text files, but the provided files are not well formed. I found a way to get the data lines and place a separator between the values, but there is a small amount of data I’m ...
0
votes
1
answer
78
views
grep stop at first match
I want to grep a file that stops at the first match, example:
${A57}${A17}${A23}${A20}${A35}${A57}${A33}${A24}${A38}${A38}${A24}${A17}${A57}${A33}${A23}${A38}${A18}${E52}${A28}${A28}${A28}${A25}${A38}$...
0
votes
2
answers
44
views
How to split a field and then to print the last element using awk
I am trying to edit a file which has this format:
field1 field2 field3 gene_id "xxxxx"; transcript_id "XM_xxxxxxxx.x"; db_xref "GeneID:102885392"; exon_number "1&...
-2
votes
1
answer
27
views
awk to separate rows from bed files depending on character
I want to separate rows by comma delimiter in one filed and keep the other information of the row. I have tab delimited files with 4 columns and a lot of rows...
Frome here:
1 13445 rs558318514 C G,...
1
vote
6
answers
107
views
AWK not able to extract the from field in the maillog
I am trying to extract some fields from the maillog and it worked fine except this below message when it failed.
May 10 07:15:04 chitraak sendmail[8558]: 34ABF4Kb008569: from=<"dexter prod&...
-2
votes
1
answer
129
views
renumbering values in .pdb file
I have one .pdb file, which correspond to peptide-protein complex. Fragment of this file looks like this:
edit: replaced the sample input with a subset of what OP provided afterwards.
ATOM 51 O ...