Skip to main content
Filter by
Sorted by
Tagged with
1 vote
2 answers
69 views

increment a column and line number value using bash

I have a tab-delimited file that contains 4 columns numbers.txt 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 In bash I need to be able to ...
Lurch's user avatar
  • 859
-3 votes
2 answers
97 views

How to compare two tab-separated files and mark 'Fail' if col6 of file1 matches col2 of file2, else 'Pass'? [closed]

I want to compare two tab-separated files (with headers) and check whether column 6 of the first file (peptide.tsv) matches exactly with column 2 of the second file (proteome.tsv). If there is an ...
shafaque's user avatar
0 votes
5 answers
95 views

Regexp/grep to find one of two identical values, depending on a heading?

I have this output on the command line: % pmset -g custom Battery Power: lidwake 1 lowpowermode 1 standbydelayhigh 86400 standbydelaylow 10800 standby 1 ...
d-b's user avatar
  • 947
0 votes
2 answers
83 views

find unique entries in one field across two CSV files with shell and friends

I have two CSV files I want to compare. The first looks like: aaaaaaaa-fb34-4e3e-aed5-eec78d02d59b,1234 bbbbbbbb-f76a-4a05-bb53-63aba9d03fe2,5678 cccccccc-e351-4d8e-b44a-080f6ccdef7d,9012 The second ...
tsj's user avatar
  • 786
1 vote
7 answers
101 views

How to use bash for loop to awk print FPAT string found in quoted variable

I cant figure out the proper FPAT to get each comma separated value. Using bash on Debian 12.5... I have a file PING.cfg with contents: NODE01="router,10.0.0.1,5" NODE02="dns1,68.6.16....
SmplJohn's user avatar
1 vote
5 answers
131 views

Put a substring of a string into a variable

I'm trying to take a substring of a string read from a file and put it in a variable. The string will look something like this: "dn: uid=myUid,ou=People,dc=mydomain,dc=furtherdomain" The ...
Eric Fetzer's user avatar
2 votes
3 answers
586 views

Bash script that takes multiple path arguments and checks if files can be successfully created there

I would like a bash shell script (for my Debian 12 system) which accepts multiple paths as arguments, and then checks if a file can be successfully created in each one. Then the result should be put ...
teeeeee's user avatar
  • 707
1 vote
1 answer
69 views

awk in pipeline to match patterns from multiple lines and print summary on trigger

There is a log file from a imaging session where I need to extract parameters that are in multiple lines preceding a certain trigger. Once the trigger is found I want to print the collected data and ...
Gert Gottschalk's user avatar
2 votes
4 answers
61 views

Calculate subtotals of the output of sort-unique command

I have a file that is generated using command | sort | uniq -c city.txt 2 mumbaiXa 3 mumbaiXb 1 mumbaiXp 5 delhiXn 4 delhiXz 1 parisXs 7 parisXt 1 parisXa 9 parisXe I am trying to split on X and get ...
shantanuo's user avatar
  • 32.2k
1 vote
3 answers
87 views

Filter the docker ps output with a Bash alias

I'm using Docker for many of our services and the command docker ps is essential for me but I hate the long and extensive output. For example in this server: [root@server docker-postgres]# docker ps ...
Fernando Iñiguez's user avatar
0 votes
6 answers
169 views

Split the word using bigram, trigram

I have this text file: worked working works tested tests find found It contains a million words without spaces. It may contain unicode characters. The longest word is "working": awk '{print ...
shantanuo's user avatar
  • 32.2k
-3 votes
4 answers
84 views

Remove all occurences of substring from variable given prefix & suffix of substring [closed]

As an example, I have a variable containing: sentence="\<Name\>Mary\</Name\> has a little lamb it's fleece as white as snow and everywhere that \<Name\>Sharon\</Name\> ...
Adi's user avatar
  • 95
1 vote
5 answers
89 views

Custom Line Number/Prefix for text file using awk/sed

Before raising the flag I tried couple of things but didn't get the desired result. I have many text files which I want prefixed with custom numbering; the first 2 lines should be prefixed 00A: , ...
Savin's user avatar
  • 27
0 votes
3 answers
193 views

awk search and replace by a string containing "\n" will add a new line instead of writing "\n"

I use Awk to search a keyword <replaceme> and replace it by a string. This string contains the chars "\n", but Awk interprets it as a new line. I really want to keep both characters, \ ...
Nico's user avatar
  • 29
-7 votes
3 answers
165 views

To move character sequences to the next line depending on the position of this line within paragraph [closed]

Here is an example text: Lorem ipsum dolor sit amet, consectetur adipiscing elit, satoru do a eiusmod tempor incididunt ut labore et dolore magna aliqua. Do b c quis autem vel eum iure reprehenderit ...
nick's user avatar
  • 141
-2 votes
2 answers
125 views

how to get summary of number of additions and deletions per commit and user for each day in git

So far I have been reading the below two responses: How to count total lines changed by a specific author in a Git repository? https://gist.github.com/eyecatchup/3fb7ef0c0cbdb72412fc But I have been ...
qwer's user avatar
  • 227
7 votes
7 answers
269 views

Count unique values in one column based on a duplicate in another column

I have a file with an identifier and a value: ABC123 111111 ABC123 111111 ABCDEF 333333 ABCDEF 111111 CCCCCC 333333 ABC123 222222 DEF123 444444 DEF123 444444 Both columns contain duplicate values, ...
Hashim Aziz's user avatar
  • 5,514
1 vote
1 answer
56 views

Using column header to write variables to replace columns from different csv files (using awk) [duplicate]

I have been working on making a bash script generic so that it would allow for input csv files with a different number of columns. I want to replace one column in a csv file with another column from a ...
LadyG's user avatar
  • 41
1 vote
0 answers
54 views

How can I replacing columns between csv files using columns headers as match in bash scripting? [duplicate]

I am new to bash scripting and I am putting together a simple script to take a column from one csv file and then use it to replace a column in another csv file. I managed to do it with awk this way ...
LadyG's user avatar
  • 41
4 votes
6 answers
138 views

Get columns from a line where there might be quotation preferrably but not necessarily in awk [duplicate]

I have data from 'speedtest-cli' that get gathered in a cronjob. 12818 "Ridge Wireless" "Cupertino, CA" 2024-08-17T17:15:02.811911Z 142.96369289624104 33.358 322418432.2265161 ...
Gert Gottschalk's user avatar
0 votes
2 answers
71 views

Exclude the last character when printing in a for loop [duplicate]

I am using linux and trying to code a loop which can take in variables and concatenates them into an equation. But how do I remove it for the last loop. printf "(%s + %s) x ", $1, $2 I want ...
Jim's user avatar
  • 27
1 vote
3 answers
351 views

A single POSIX-Extended regex to match all shortest substrings that start with “a”, end with “b” and do not contain any element of the set of strings

Given a string, is it possible to write a single POSIX-Extended regular expression to match all shortest substrings that start with a and end with b, but do not contain cb, cd or fg? I want to use ...
lyrically wicked's user avatar
2 votes
1 answer
64 views

GNU's AWK - Splitting Multibyte Character Strings w/Patsplit vs Split

I'd like to know if the following behavior is a reliable: echo 😄 | gawk '{split($0,array,"."); print array[1] length(array);}' Output is: 😄1 vs echo 😄 | gawk '{patsplit($0,array,".&...
Adam D.'s user avatar
  • 245
0 votes
3 answers
108 views

Recombine data with multiple different delimiters [closed]

There is an array of data: https://example.com:description of the site/application:category http://example.com:description of the site/application:category android://package name:description of the ...
Fresh's user avatar
  • 19
0 votes
1 answer
109 views

Compare the column 2 of File 1 with column 4 and 5 of file 2

I have a tab delimited file_1 NC_025 4569 . KX838946.2 NC_025 16546 . KJ641660.1 NC_025 11996 . KX932454.2 file_2 NC_025.1 RefSeq gene 5690 7513 ....
Ravi Saroch's user avatar
-2 votes
1 answer
76 views

How to Rearrange the column order of a csv file [duplicate]

Am trying to rearrange the column order of below sample data , USI,CFG LEI,Counterparty LEI,UPI,Product Class,Execution venue,MTM,Currency 1,Currency 2,Notional amount 1,Notional amount 2,Exchange ...
Loganayaki mahalingam's user avatar
3 votes
4 answers
132 views

How to compare 2 file and print the first match of the string until the blank line

I need to do a backup of all users that have been deleted so I have 1 file for the backup and another file for the list of users to backup. I want it use awk to retrieve the details and would like to ...
Cj Calaguian's user avatar
1 vote
2 answers
69 views

Recursive search using awk but with pattern input from a file

How to modify this Script recursive search using awk script to replace by a file with lines of patterns to search? For example. find . -type f -exec awk '/FILE_WITH_LINES_TO_SEARCH.txt/{print ...
Kalasni's user avatar
  • 61
3 votes
4 answers
96 views

How to get strings within square brackets [] in the declarations line and their related import strings

In a text file which has the line beginning with the "declarations:" string, replace "declarations" with "imports", replace "AppComponent" or "...
albertkao9's user avatar
0 votes
1 answer
61 views

Extracting and formatting data from auth.log file using Bash [closed]

I need to extract authorization data from the auth.log file. I also need to format this data so that it is easy to view. In my case, it is several columns: Date Time User Action IP. I already have a ...
phantaserrr's user avatar
2 votes
7 answers
155 views

How to change some columns in CSV file

I have a file that looks like this: CNPJ,73811717000167,ABC,04886747000140,ATIVO,2020-04-22T16:21:47.758-03:00,null,2020-04-22T16:21:47.758-03:00,2020-04-22T16:21:47.758-03:00 CNPJ,71179825000180,ABC,...
Cristiana valente's user avatar
2 votes
3 answers
81 views

awk - Character Class Regex gsub Changes output

Parsing the following with awk: $> df -h / Filesystem Size Used Avail Use% Mounted on rootfs 476G 370G 106G 78% / If I use an explicit match for the G's on the values, it works ...
Adam D.'s user avatar
  • 245
5 votes
1 answer
162 views

What is the meaning of "@" symbol before awk regex

This it taken from the "awk-exercises". For the input file patterns.txt, filter lines containing three or more occurrences of "ar" and replace the last but second "ar" ...
mr.Nobody's user avatar
0 votes
2 answers
67 views

How to rename the CSV file based upon the column name inside the file using shell script

Let's assume I have list of CSV files under the folder called /path/RAS, each CSV file is going to carry the information about each customer. Let's take one customer file as example: Customer_Name,...
Loganayaki mahalingam's user avatar
-2 votes
1 answer
69 views

How to replace a multiline block of code by another one delimited by specified char [duplicate]

Here is a generic header of bash files. #!/bin/bash # ##################################### # # # {config_} # # ...
jcdole's user avatar
  • 65
0 votes
7 answers
182 views

Splitting and substitution by comma is not working in shell script

I have a string z="job_name=market_scenario,channel=GOOGLE FACEBOOK INSTAGRAM,time_limit=60sec" Now I want the data in the below format: job_name=market_scenario channel=GOOGLE,FACEBOOK,...
abhi pratap's user avatar
-1 votes
4 answers
130 views

How to append some static content within a single cell of an existing CSV file

Let's assume I need to append the below content within the first cell (not be row by row) of the existing CSV file which has details of the customer. How can I achieve it? Content to be added: "...
Loganayaki mahalingam's user avatar
0 votes
5 answers
137 views

How do I delete all the lines that match and one after each of them?

I have a large file and a list of my specific strings. The output should not contain my specific lines and one more after each of them. 2 consecutive matches are impossible due to structure of file ...
Женя Гончаров's user avatar
0 votes
3 answers
64 views

passing "-e" as a parameter to awk is having unexpected results

I have short Bash script that will not pass "-e " to Awk: str="Morgenstern" ex = "-e " for i in {1..11} do echo $(echo $str| awk -v vv="$i" -v xx="$ex&...
C0ppert0p's user avatar
  • 700
1 vote
5 answers
112 views

How to add headers to a CSV file which already has some headers in it

My requirement it to add the new headers to an existing CSV file which already has some headers in the fist line. I need to add the new headers at the end of the existing headers. The value of the new ...
Loganayaki mahalingam's user avatar
0 votes
1 answer
56 views

Grep a row and retain only matching rows based on a specific column in bash [closed]

I'm working on a project where I need to extract specific gene neighborhood information from a dataset based on protein accession IDs. I have a series of .acc files containing protein accession IDs, ...
Rohan Nath's user avatar
-1 votes
3 answers
115 views

How to find strings in multiple lines of txt file then add all to new csv line

I have thousands of txt files that were converted from emails that included a "Contact Us" form. I need to find/select strings in the second "column", but the number of lines to ...
Aidbfbx's user avatar
1 vote
4 answers
178 views

How to properly escape single quotes in filenames?

I have files with single quotes in their name, say o'a. I want to process them in awk (gawk-5.3.0): (fd is an alternative to find). Please don't tell me about fd -x option, my command is snippet of a ...
jgran's user avatar
  • 1,234
3 votes
3 answers
72 views

Explanation about awk matching a pattern between a range of lines

I have an Awk one-liner that, while I understand what is being accomplished at a high level, I don't fully understand how Awk is accomplishing the task I give it. I'm asking Awk to give me a range of ...
Terucin's user avatar
  • 31
0 votes
0 answers
38 views

capture hash value from git log --oneline --graph using regex [duplicate]

When running git log --oneline --graph you can get something like this * 7e5dbb0 Merge |\ | * c873a96 My | * f5817ba Amazing * | f6a34ac Feature |/ * f5a51ce Other stuff I want to take that and ...
Net-worker's user avatar
1 vote
4 answers
123 views

Awk /Sed or TCL script to search series of patterns

I am having one File as stated below File 1 Module1 (a dd ddd ) jdjd ikdnd kdndn didkdn; jdjdjl kdkdk kdkd lkd; Ann {"Load DR"} ; V { REST_TMS=1 ; REST_TDI = 0 REST_TCK=1 ; ...
kshitij kulshreshtha's user avatar
7 votes
7 answers
285 views

gawk hangs when using a regex for RS combined with reading a continuous stream from stdin

I'm streaming data using netcat and piping the output to gawk. Here is an example byte sequence that gawk will receive: =AAAA;=BBBB;;CCCC==DDDD; The data includes nearly any arbitrary characters, but ...
user12280249's user avatar
1 vote
2 answers
73 views

How to create AWK scripts with shebang that allows in-place modifications?

I want create a awk script with a shebang such that ./main.awk text.txt performs in-place modification in text.txt. Inspired by a previous solution, I tried something like #!/usr/bin/awk -i inplace -...
Rubem Pacelli's user avatar
0 votes
2 answers
86 views

Keep only species name in FASTA header

I tried coding in Bash with the Awk and sed commands but didn't get the desired output. I have a text file with the following contents: >AC201869.46386.47908 Regiella insecticola ...
Abhisek's user avatar
  • 27
1 vote
2 answers
135 views

Why does the “match” function not work in this particular situation?

I have the 57-byte text file (in UTF-8). It can be generated by the following command: echo '...
lyrically wicked's user avatar

1
2 3 4 5
664