33,154
questions
1
vote
2
answers
69
views
increment a column and line number value using bash
I have a tab-delimited file that contains 4 columns
numbers.txt
0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
In bash I need to be able to ...
-3
votes
2
answers
97
views
How to compare two tab-separated files and mark 'Fail' if col6 of file1 matches col2 of file2, else 'Pass'? [closed]
I want to compare two tab-separated files (with headers) and check whether column 6 of the first file (peptide.tsv) matches exactly with column 2 of the second file (proteome.tsv). If there is an ...
0
votes
5
answers
95
views
Regexp/grep to find one of two identical values, depending on a heading?
I have this output on the command line:
% pmset -g custom
Battery Power:
lidwake 1
lowpowermode 1
standbydelayhigh 86400
standbydelaylow 10800
standby 1
...
0
votes
2
answers
83
views
find unique entries in one field across two CSV files with shell and friends
I have two CSV files I want to compare. The first looks like:
aaaaaaaa-fb34-4e3e-aed5-eec78d02d59b,1234
bbbbbbbb-f76a-4a05-bb53-63aba9d03fe2,5678
cccccccc-e351-4d8e-b44a-080f6ccdef7d,9012
The second ...
1
vote
7
answers
101
views
How to use bash for loop to awk print FPAT string found in quoted variable
I cant figure out the proper FPAT to get each comma separated value. Using bash on Debian 12.5... I have a file PING.cfg with contents:
NODE01="router,10.0.0.1,5"
NODE02="dns1,68.6.16....
1
vote
5
answers
131
views
Put a substring of a string into a variable
I'm trying to take a substring of a string read from a file and put it in a variable. The string will look something like this:
"dn: uid=myUid,ou=People,dc=mydomain,dc=furtherdomain"
The ...
2
votes
3
answers
586
views
Bash script that takes multiple path arguments and checks if files can be successfully created there
I would like a bash shell script (for my Debian 12 system) which accepts multiple paths as arguments, and then checks if a file can be successfully created in each one. Then the result should be put ...
1
vote
1
answer
69
views
awk in pipeline to match patterns from multiple lines and print summary on trigger
There is a log file from a imaging session where I need to extract parameters that are in multiple lines preceding a certain trigger. Once the trigger is found I want to print the collected data and ...
2
votes
4
answers
61
views
Calculate subtotals of the output of sort-unique command
I have a file that is generated using command | sort | uniq -c
city.txt
2 mumbaiXa
3 mumbaiXb
1 mumbaiXp
5 delhiXn
4 delhiXz
1 parisXs
7 parisXt
1 parisXa
9 parisXe
I am trying to split on X and get ...
1
vote
3
answers
87
views
Filter the docker ps output with a Bash alias
I'm using Docker for many of our services and the command docker ps is essential for me but I hate the long and extensive output.
For example in this server:
[root@server docker-postgres]# docker ps
...
0
votes
6
answers
169
views
Split the word using bigram, trigram
I have this text file:
worked
working
works
tested
tests
find
found
It contains a million words without spaces. It may contain unicode characters.
The longest word is "working":
awk '{print ...
-3
votes
4
answers
84
views
Remove all occurences of substring from variable given prefix & suffix of substring [closed]
As an example, I have a variable containing:
sentence="\<Name\>Mary\</Name\> has a little lamb it's fleece as white as snow and everywhere that \<Name\>Sharon\</Name\> ...
1
vote
5
answers
89
views
Custom Line Number/Prefix for text file using awk/sed
Before raising the flag I tried couple of things but didn't get the desired result.
I have many text files which I want prefixed with custom numbering;
the first 2 lines should be prefixed 00A: , ...
0
votes
3
answers
193
views
awk search and replace by a string containing "\n" will add a new line instead of writing "\n"
I use Awk to search a keyword <replaceme> and replace it by a string. This string contains the chars "\n", but Awk interprets it as a new line. I really want to keep both characters, \ ...
-7
votes
3
answers
165
views
To move character sequences to the next line depending on the position of this line within paragraph [closed]
Here is an example text:
Lorem ipsum dolor sit amet, consectetur adipiscing elit, satoru do a
eiusmod tempor incididunt ut labore et dolore magna aliqua. Do b c
quis autem vel eum iure reprehenderit ...
-2
votes
2
answers
125
views
how to get summary of number of additions and deletions per commit and user for each day in git
So far I have been reading the below two responses:
How to count total lines changed by a specific author in a Git repository?
https://gist.github.com/eyecatchup/3fb7ef0c0cbdb72412fc
But I have been ...
7
votes
7
answers
269
views
Count unique values in one column based on a duplicate in another column
I have a file with an identifier and a value:
ABC123 111111
ABC123 111111
ABCDEF 333333
ABCDEF 111111
CCCCCC 333333
ABC123 222222
DEF123 444444
DEF123 444444
Both columns contain duplicate values, ...
1
vote
1
answer
56
views
Using column header to write variables to replace columns from different csv files (using awk) [duplicate]
I have been working on making a bash script generic so that it would allow for input csv files with a different number of columns.
I want to replace one column in a csv file with another column from a ...
1
vote
0
answers
54
views
How can I replacing columns between csv files using columns headers as match in bash scripting? [duplicate]
I am new to bash scripting and I am putting together a simple script to take a column from one csv file and then use it to replace a column in another csv file.
I managed to do it with awk this way
...
4
votes
6
answers
138
views
Get columns from a line where there might be quotation preferrably but not necessarily in awk [duplicate]
I have data from 'speedtest-cli' that get gathered in a cronjob.
12818 "Ridge Wireless" "Cupertino, CA" 2024-08-17T17:15:02.811911Z 142.96369289624104 33.358 322418432.2265161 ...
0
votes
2
answers
71
views
Exclude the last character when printing in a for loop [duplicate]
I am using linux and trying to code a loop which can take in variables and concatenates them into an equation. But how do I remove it for the last loop.
printf "(%s + %s) x ", $1, $2
I want ...
1
vote
3
answers
351
views
A single POSIX-Extended regex to match all shortest substrings that start with “a”, end with “b” and do not contain any element of the set of strings
Given a string, is it possible to write a single POSIX-Extended regular expression to match all shortest substrings that start with a and end with b, but do not contain cb, cd or fg? I want to use ...
2
votes
1
answer
64
views
GNU's AWK - Splitting Multibyte Character Strings w/Patsplit vs Split
I'd like to know if the following behavior is a reliable:
echo 😄 | gawk '{split($0,array,"."); print array[1] length(array);}'
Output is: 😄1
vs
echo 😄 | gawk '{patsplit($0,array,".&...
0
votes
3
answers
108
views
Recombine data with multiple different delimiters [closed]
There is an array of data:
https://example.com:description of the site/application:category
http://example.com:description of the site/application:category
android://package name:description of the ...
0
votes
1
answer
109
views
Compare the column 2 of File 1 with column 4 and 5 of file 2
I have a tab delimited file_1
NC_025 4569 . KX838946.2
NC_025 16546 . KJ641660.1
NC_025 11996 . KX932454.2
file_2
NC_025.1 RefSeq gene 5690 7513 ....
-2
votes
1
answer
76
views
How to Rearrange the column order of a csv file [duplicate]
Am trying to rearrange the column order of below sample data ,
USI,CFG LEI,Counterparty LEI,UPI,Product Class,Execution venue,MTM,Currency 1,Currency 2,Notional amount 1,Notional amount 2,Exchange ...
3
votes
4
answers
132
views
How to compare 2 file and print the first match of the string until the blank line
I need to do a backup of all users that have been deleted so I have 1 file for the backup and another file for the list of users to backup. I want it use awk to retrieve the details and would like to ...
1
vote
2
answers
69
views
Recursive search using awk but with pattern input from a file
How to modify this Script recursive search using awk script to replace by a file with lines of patterns to search?
For example.
find . -type f -exec awk '/FILE_WITH_LINES_TO_SEARCH.txt/{print ...
3
votes
4
answers
96
views
How to get strings within square brackets [] in the declarations line and their related import strings
In a text file which has the line beginning with the "declarations:" string,
replace "declarations" with "imports",
replace "AppComponent" or "...
0
votes
1
answer
61
views
Extracting and formatting data from auth.log file using Bash [closed]
I need to extract authorization data from the auth.log file. I also need to format this data so that it is easy to view. In my case, it is several columns: Date Time User Action IP.
I already have a ...
2
votes
7
answers
155
views
How to change some columns in CSV file
I have a file that looks like this:
CNPJ,73811717000167,ABC,04886747000140,ATIVO,2020-04-22T16:21:47.758-03:00,null,2020-04-22T16:21:47.758-03:00,2020-04-22T16:21:47.758-03:00
CNPJ,71179825000180,ABC,...
2
votes
3
answers
81
views
awk - Character Class Regex gsub Changes output
Parsing the following with awk:
$> df -h /
Filesystem Size Used Avail Use% Mounted on
rootfs 476G 370G 106G 78% /
If I use an explicit match for the G's on the values, it works ...
5
votes
1
answer
162
views
What is the meaning of "@" symbol before awk regex
This it taken from the "awk-exercises".
For the input file patterns.txt, filter lines containing three or more
occurrences of "ar" and replace the last but second "ar" ...
0
votes
2
answers
67
views
How to rename the CSV file based upon the column name inside the file using shell script
Let's assume I have list of CSV files under the folder called /path/RAS, each CSV file is going to carry the information about each customer.
Let's take one customer file as example:
Customer_Name,...
-2
votes
1
answer
69
views
How to replace a multiline block of code by another one delimited by specified char [duplicate]
Here is a generic header of bash files.
#!/bin/bash
#
#####################################
# #
# {config_} #
# ...
0
votes
7
answers
182
views
Splitting and substitution by comma is not working in shell script
I have a string
z="job_name=market_scenario,channel=GOOGLE FACEBOOK INSTAGRAM,time_limit=60sec"
Now I want the data in the below format:
job_name=market_scenario
channel=GOOGLE,FACEBOOK,...
-1
votes
4
answers
130
views
How to append some static content within a single cell of an existing CSV file
Let's assume I need to append the below content within the first cell (not be row by row) of the existing CSV file which has details of the customer. How can I achieve it?
Content to be added:
"...
0
votes
5
answers
137
views
How do I delete all the lines that match and one after each of them?
I have a large file and a list of my specific strings. The output should not contain my specific lines and one more after each of them. 2 consecutive matches are impossible due to structure of file ...
0
votes
3
answers
64
views
passing "-e" as a parameter to awk is having unexpected results
I have short Bash script that will not pass "-e " to Awk:
str="Morgenstern"
ex = "-e "
for i in {1..11}
do
echo $(echo $str| awk -v vv="$i" -v xx="$ex&...
1
vote
5
answers
112
views
How to add headers to a CSV file which already has some headers in it
My requirement it to add the new headers to an existing CSV file which already has some headers in the fist line. I need to add the new headers at the end of the existing headers. The value of the new ...
0
votes
1
answer
56
views
Grep a row and retain only matching rows based on a specific column in bash [closed]
I'm working on a project where I need to extract specific gene neighborhood information from a dataset based on protein accession IDs. I have a series of .acc files containing protein accession IDs, ...
-1
votes
3
answers
115
views
How to find strings in multiple lines of txt file then add all to new csv line
I have thousands of txt files that were converted from emails that included a "Contact Us" form. I need to find/select strings in the second "column", but the number of lines to ...
1
vote
4
answers
178
views
How to properly escape single quotes in filenames?
I have files with single quotes in their name, say o'a. I want to process them in awk (gawk-5.3.0):
(fd is an alternative to find).
Please don't tell me about fd -x option, my command is snippet of a ...
3
votes
3
answers
72
views
Explanation about awk matching a pattern between a range of lines
I have an Awk one-liner that, while I understand what is being accomplished at a high level, I don't fully understand how Awk is accomplishing the task I give it. I'm asking Awk to give me a range of ...
0
votes
0
answers
38
views
capture hash value from git log --oneline --graph using regex [duplicate]
When running git log --oneline --graph you can get something like this
* 7e5dbb0 Merge
|\
| * c873a96 My
| * f5817ba Amazing
* | f6a34ac Feature
|/
* f5a51ce Other stuff
I want to take that and ...
1
vote
4
answers
123
views
Awk /Sed or TCL script to search series of patterns
I am having one File as stated below
File 1
Module1 (a dd ddd )
jdjd ikdnd kdndn didkdn;
jdjdjl kdkdk kdkd lkd;
Ann {"Load DR"} ;
V { REST_TMS=1 ;
REST_TDI = 0
REST_TCK=1 ;
...
7
votes
7
answers
285
views
gawk hangs when using a regex for RS combined with reading a continuous stream from stdin
I'm streaming data using netcat and piping the output to gawk. Here is an example byte sequence that gawk will receive:
=AAAA;=BBBB;;CCCC==DDDD;
The data includes nearly any arbitrary characters, but ...
1
vote
2
answers
73
views
How to create AWK scripts with shebang that allows in-place modifications?
I want create a awk script with a shebang such that
./main.awk text.txt
performs in-place modification in text.txt. Inspired by a previous solution, I tried something like
#!/usr/bin/awk -i inplace -...
0
votes
2
answers
86
views
Keep only species name in FASTA header
I tried coding in Bash with the Awk and sed commands but didn't get the desired output. I have a text file with the following contents:
>AC201869.46386.47908 Regiella insecticola
...
1
vote
2
answers
135
views
Why does the “match” function not work in this particular situation?
I have the 57-byte text file (in UTF-8). It can be generated by the following command:
echo '...