Questions tagged [awk]

AWK is an interpreted programming language designed for text processing and typically used as a data extraction and reporting tool. AWK is used largely with Unix systems.

Filter by
Sorted by
Tagged with
-1 votes
0 answers
16 views

Unbelievable: Awk rounding a float to integer got weird result: 123.12*100 -> 12312(good) but 1234.12*100 -> 123411(bad)

awk '{printf "%d\n", (1234.12*100)}' <<<"" outputs 123411 It is supposed to be 123412. Awk version: GNU Awk 4.1.4, API: 1.1 (GNU MPFR 4.0.1, GNU MP 6.1.2) What am I ...
osexp2000's user avatar
  • 2,840
0 votes
0 answers
11 views

How i can delete duplicate lines but leave the line where field have particular value

I need to delete duplicate lines in a pipe separated text file and leave the line where field 11 and 16 contain the value 9. how can I do? for example if I have 3 records L|20230502|0390286|F|0|0000|...
Michele Santucci's user avatar
-2 votes
0 answers
40 views

awk command: how to delete lines from file with "only blank characters" or only <tab> or <cr>? [closed]

I made an awk script to clean up commented files (gforth in emacs), See the code below. However, few lines with following characters are still in the output file (should be gone): (backslash) then (...
floppy_molly's user avatar
1 vote
4 answers
64 views

Is there a better way to parse the output of a command that produces a string with multiple columns than using awk '{print substr($0, 68)}'? [closed]

I run a CLI command that outputs various database values. One CLI command example is: show attributes attr=0x1006e mh=0x12003ea | awk '{print substr($0, 68)}' | grep -v Value The output looks like ...
DanL's user avatar
  • 21
0 votes
2 answers
74 views

How to fill the values to a shell script from tsv file and create new file from 1st column value

I have a data.tsv file with 100 entries, as below ColumnA ColumnB ColumnC ColumnD ColumnE Cell 31 Cell 4 Cell 3 Cell 5 Cell 8 Cell 21 Cell 2 Cell 5 Cell 6 Cell 9 and I have a template.in script ...
devbasic's user avatar
0 votes
2 answers
66 views

Find nearest range higher and lower inside another range

I have rewritten this question many times, as while writing it out, I have come up with some ideas and have managed to get a very slow, but working solution. It seems writing out what one needs to do ...
DJ Champion's user avatar
-2 votes
0 answers
32 views

Append multiline from a file using AWK after a multiline patter search [closed]

I have a file which is of 400 Lines. I need to search multiple lines which is a pattern and then after the pattern it needs to append few more lines from a file. The command i am using is awk -v RS='^$...
Shubhro's user avatar
  • 69
4 votes
8 answers
91 views

Shell command to add double quotes for number surrounded by dots in property keys

I have following property keys connection.party= 0.0.0.0 remote.app.328.port= 5432 remote.server.url="https://someurl.com?q=78.78" remote.conncetion.7="basic" remote.52.base= "...
Débora's user avatar
  • 5,836
-1 votes
4 answers
71 views

Calculating the median value from a TSV file for each month [closed]

I have a TSV file and I am trying to perform some analysis. I have a month column and the data is numbers 1-12 which corresponds to the month (Jan=1, Feb=2 etc). I am trying to setup a counter so that ...
Spoku's user avatar
  • 49
0 votes
0 answers
63 views

How do you issue a 'break' command in a nested while/do/done containing multiple loops?

#!/bin/sh set -x ######################################################################### # This script will export a listing of all devices and where each device # lives (...
DanL's user avatar
  • 21
-1 votes
0 answers
25 views

awk single regular expression in matching two different patterns [duplicate]

My awk version is GNU Awk 5.0.1, API: 2.0 (GNU MPFR 4.0.2, GNU MP 6.2.0). Both echo '{{' | awk '/{{2}/ { print "foo" }' and echo '{{2}' | awk '/{{2}/ { print "foo" }' are ...
Sourav Kannantha B's user avatar
2 votes
2 answers
63 views

awk not finding a string value with special characters in a csv file

I'm just trying to find a way to check if a string value exists in a csv column using awk command. It works fine except for the strings which contains special characters like ( or [. It worked for ...
francescxrs's user avatar
-1 votes
0 answers
41 views

BASH Append a backslash before a space in a string and show the backslash with the space [closed]

Good Day A file has two spaces in its name. /storage/vbk-2023/2023-01-voi-decrypt/DAV202/2023-01-voi-decrypt - DAV202.vbm I am trying to put two backslash at the two positions, as below, ...
Nash A's user avatar
  • 75
2 votes
1 answer
66 views

Matching a regex metacharacter literally [closed]

My understanding of Regex's in AWK is that in order to match a Regex metacharacter literally (For example: +,$,^,*,etc) You must escape them, like so: awk -F '\\+' 'program here' However I've noticed ...
the_slug's user avatar
3 votes
5 answers
114 views

How do I remove the last 2 columns from a tsv file?

I am coding using bash on terminal through a docker container on my mac. I am struggling to figure out how to remove the last 2 columns on my TSV file. It has 7 total and the last 2 are not needed for ...
Spoku's user avatar
  • 49
0 votes
0 answers
26 views

awk - turning date data into schedule [closed]

I have the following data: Body,23/04/23,14/05/23,,,,,, Tony,23/04/23,14/05/23,04/06/23,25/06/23,,,, Sat,25/04/23,16/05/23,06/06/23,27/06/23,,,, Gil,25/04/23,16/05/23,06/06/23,,,,, Chan,25/04/23,13/05/...
Christian Hick's user avatar
-2 votes
1 answer
54 views

How to inject data through awk from CSV to Redis [closed]

I'm trying to inject data from csv file to Redis. If I run the script in the terminal, I get a JSON format, but I do not know how to insert this data into Redis. I try using SET at the beginning and | ...
CristianG's user avatar
5 votes
5 answers
88 views

Extract 2 different and non-consecutive lines from text without creating temporary text files

I'm a musician and do a little programming on the side, so my abilities are limited, please forgive me. Sometimes I need the LUFS level of multiple audio files and the quickest way (for me...) was to ...
tanzpartner's user avatar
0 votes
1 answer
35 views

chnaging a specific column in a specific row with for loop

I have a file with following content ### beginning of executable commands ### load the necessary module files module load GCC/11.3.0 OpenMPI/4.1.4 CP2K/9.1 ### your program goes here (a.out is an ...
Chintu 's user avatar
2 votes
3 answers
65 views

How extract strings in first and last set of parenthasis?

I have a file: queueN.txt, that looks like this: uid=524357(aali95) gid=524357(aali95) groups=524357(aali95),611684(com_perkinsd),695433(bios420) uid=524534(aalman4) gid=524534(aalman4) groups=524534(...
anikaM's user avatar
  • 379
1 vote
2 answers
77 views

AWK how to permanently remove a date field from each line of a datafile when date value is older than x days?

I have a todo list in todo.txt format. I'd like to find the starting time ('t:2023-05-18') for all tasks with a date older than 10 days and remove the field from each matching line. Sample Input Data (...
Tony Barganski's user avatar
1 vote
0 answers
57 views

using awk/sed to format log output into required CSV format [closed]

Hopefully this makes more sense, What I'm trying to achieve is two things, firstly to format the data in CSV format and secondly to perform a match on the node id to return if its patched as YES/NO ...
dcepticon's user avatar
0 votes
2 answers
59 views

replace text in html with sed or awk

Could you please advice how to replace text in the html tags? <link href="*" rel="icon" type="image/x-icon" /> I want to get instead of example above: <link ...
macder's user avatar
  • 131
3 votes
1 answer
48 views

Grab Nth occurrance of each element with AWK

This is the file: AAACGCTGTGTCATTG-1-pere,1,2 AAACGCTTTGTCATTG-1-pere,3,6 AAACGCTATGTCATTG-1-pere,3,4 AAACGCTCTGTCATTG-1-mele,2,1 AAACGCTFTGTCATTG-1-pere,5,8 AAACGCTHTGTCATTG-1-mele,5,3 ...
pino's user avatar
  • 83
3 votes
3 answers
66 views

if else command in IBM AIX OS

I would check in the /etc/security/passwd file if a password is assigned to root user. We have to types of results: root: password = * or root: password = {ssha512}*********** i wrote ...
medisamm's user avatar
  • 197
-1 votes
2 answers
88 views

Parsing a nested cell in a CSV file using the bash terminal or R

Attempting to create a scaled-down approach to reading in a LARGE (>150GB) CSV file into an R script by chunking it into smaller bits which can be read in sequentially. Problem is, one of the ...
TDeramus's user avatar
0 votes
1 answer
79 views

In Hadoop HDFS, delete several files older than x days and with spaces in the name (Not like UNIX)

I have hundreds of thousands of files in a hadoop directory and I need to debug them. I'm looking to delete files that are more than 3 months old and I'm trying to delete in batches of a thousand ...
alejomarchan's user avatar
-2 votes
1 answer
25 views

awk printf printing extra commas [closed]

My script, part of creating html tables in email using awk, printing additional commas (,) `if (m_premium > mp) {printf ("%s,%.2f,%s\n","<td bgcolor=pink>",m_premium,"...
user1400953's user avatar
1 vote
7 answers
78 views

If the current line has the search string, do not print it and its previous line

How to do this? If the current line has the string "9999-01-31", do not print it and its previous line. Otherwise print. e.g. in.txt: Test1 end1 Date1 : 2023-01-12 Date2 : 2023-07-23 Start ...
albertkao9's user avatar
-1 votes
1 answer
33 views

Delete columns, not just empty them [duplicate]

I'm trying to delete the last 2 columns of a csv file with awk. I have tried using: x=$(awk -v FPAT='([^,]*)|(\"[^\"]+\")' -v OFS=',' '{$10=$11=""; print $0}' ${filename} > ...
Marco's user avatar
  • 1
0 votes
2 answers
55 views

looking to convert first name last name to first initial last name from a getent output

I have to convert this. getent passwd euser5 euser5:x:12345:20:Ester Jones:/home/euser5:/bin/csh so far i have this. getent passwd euser5|cut -d: -f1,5 |sed 's/:/,/'|tr '[:upper:]' '[:lower:]' euser5,...
keith's user avatar
  • 13
-2 votes
0 answers
21 views

How to print increment number when we use print command in xterm [closed]

i have to use below command and i need to add increment number in name sonu can anyone please help. not able to get the
sohin borad's user avatar
0 votes
1 answer
34 views

Using awk to find/replace the string in text file from defined variable

I have a input text file as below. ... ... DataColumn No X Y W H Kind Ch Count1 Type Count2; DataList 10 1 -45268.1732 50408.958800 0.0900 0.8998 0000000000000600 00800000 2 6 ...
Jiwoo Sun's user avatar
-3 votes
1 answer
49 views

keep N number of duplicate rows (based on column 1) [closed]

I have a csv with this first column of indexes: TTTGACTCATCCGATA-1-H0025_apex TTTGATCTCAAGGCTT-1-H0025_apex TTTGGAGAGTTTGAGA-1-H0025_apex TTTGGAGGTAAGATAC-1-H0025_apex TTTGGTTCAAGTGACG-1-H0025_apex ...
pino's user avatar
  • 83
-2 votes
1 answer
60 views

AWK to handle bed files

I would like to grep and separate fields from bed files to generate a new bed file with these new arranged data. I would go from here: 1 15903 rs557514207 G G,A RS=557514207;RSPOS=15903;...
DaN's user avatar
  • 3
2 votes
1 answer
59 views

Count columns from sacct output with awk in bash

I work on a slurm cluster and I want to find how much cpu and memory I am currently using. My main problem is distinguishing between 8G and 8Gc for memory usage. (the former is the raw required memory,...
Qant123's user avatar
  • 141
0 votes
4 answers
83 views

Word count command returning incorrect row count for CSV files

I am running the following command to count the number of rows in a CSV file: wc -l filename.csv Sometimes the row count is correct and other times it is 1 less than the actual count. All of the ...
Captain Caveman's user avatar
2 votes
2 answers
67 views

How can I make awk output more readable?

With this code I get the students with the highest and lowest value. But I can't find how to turn it into a more readable output. Code awk -F ';' 'NR==1 { min = max = $2; mina = maxa = $1 " "...
A. Cedano's user avatar
  • 477
0 votes
1 answer
51 views

How to recursively replace a string in all files of a folder with the contents of specific file outside the folder using bash script?

Problem: I have a folder (Essays) containing many nested sub-directories and ".txt" files. Only some files within this folder contain the string '[[RepeatedPara]]' somewhere among the ...
YourPalNurav's user avatar
  • 1,168
0 votes
3 answers
87 views

Delete lines in blocklist file, where the end of those lines match an entry from an allowlist file (Using Dnsmasq syntax)

This is a modification of a question I've previously asked, this time to account for Dnsmasq syntax in the blocklist. I'm trying to delete lines in an blocklist file, but only if the end of the ...
Wizballs's user avatar
1 vote
2 answers
85 views

How to get max and min values in linux shell with multiple possibilities?

This shell script gets me the max and min values, but I would like to get those values and the associated students when there are repeats. Code: #!/bin/sh Notamax=0 Notamin=0 alumneMax='' alumneMin='' ...
A. Cedano's user avatar
  • 477
0 votes
2 answers
71 views

How can I update an Awk command to work with a newer version of MacOS? [closed]

I am using the following Awk command which is based on this Stack Exchange post: tail -n +2 *.csv | sort -t',' -k2 | awk -F',' '$2~/^[[:space:]]*$/{next} {sub(/\r$/,"")} $2!=prev{close(out); ...
big_smile's user avatar
  • 1,487
0 votes
3 answers
86 views

output of awk if statement is wrong

I have a file with thousand's of rows and several columns like the next one: 38.13,38.05,2003-04-13,2.6,12,T22:47:52,Md and I want to change the value of column 4 according to the value of column 7 ...
meocomandate's user avatar
2 votes
1 answer
51 views

Need to get file name, line number, match position of a pattern in a file [duplicate]

I have a requirement where I have to get the file name, line number, match position (byte offset) of a pattern in a file. I am using the grep command for this and so far am able to get the file name ...
Lazy Enthusiast's user avatar
0 votes
4 answers
94 views

Combine grep and sed to extract lines of data from a text file

I must extract data from text files, but the provided files are not well formed. I found a way to get the data lines and place a separator between the values, but there is a small amount of data I’m ...
Donald Alexandre's user avatar
0 votes
1 answer
78 views

grep stop at first match

I want to grep a file that stops at the first match, example: ${A57}${A17}${A23}${A20}${A35}${A57}${A33}${A24}${A38}${A38}${A24}${A17}${A57}${A33}${A23}${A38}${A18}${E52}${A28}${A28}${A28}${A25}${A38}$...
Nemsis's user avatar
  • 3
0 votes
2 answers
44 views

How to split a field and then to print the last element using awk

I am trying to edit a file which has this format: field1 field2 field3 gene_id "xxxxx"; transcript_id "XM_xxxxxxxx.x"; db_xref "GeneID:102885392"; exon_number "1&...
Nico64's user avatar
  • 161
-2 votes
1 answer
27 views

awk to separate rows from bed files depending on character

I want to separate rows by comma delimiter in one filed and keep the other information of the row. I have tab delimited files with 4 columns and a lot of rows... Frome here: 1 13445 rs558318514 C G,...
DaN's user avatar
  • 3
1 vote
6 answers
107 views

AWK not able to extract the from field in the maillog

I am trying to extract some fields from the maillog and it worked fine except this below message when it failed. May 10 07:15:04 chitraak sendmail[8558]: 34ABF4Kb008569: from=<"dexter prod&...
isilia's user avatar
  • 361
-2 votes
1 answer
129 views

renumbering values in .pdb file

I have one .pdb file, which correspond to peptide-protein complex. Fragment of this file looks like this: edit: replaced the sample input with a subset of what OP provided afterwards. ATOM 51 O ...
skywalker's user avatar

1
2 3 4 5
651