80

Can we use shell variables in AWK like $VAR instead of $1, $2? For example:

UL=(AKHIL:AKHIL_NEW,SWATHI:SWATHI_NEW)

NUSR=`echo ${UL[*]}|awk -F, '{print NF}'`
echo $NUSR
echo ${UL[*]}|awk -F, '{print $NUSR}'

Actually am an oracle DBA we get lot of import requests. I'm trying to automate it using the script. The script will find out the users in the dump and prompt for the users to which dump needs to be loaded.

Suppose the dumps has two users AKHIL, SWATHI (there can be may users in the dump and i want to import more number of users). I want to import the dumps to new users AKHIL_NEW and SWATHI_NEW. So the input to be read some think like AKHIL:AKHIL_NEW,SWATHI:SWATHI_NEW.

First, I need to find the Number of users to be created, then I need to get new users i.e. AKHIL_NEW,SWATHI_NEW from the input we have given. So that I can connect to the database and create the new users and then import. I'm not copying the entire code: I just copied the code from where it accepts the input users.

UL=(AKHIL:AKHIL_NEW,SWATHI:SWATHI_NEW) ## it can be many users like     USER1:USER1_NEW,USER2_USER2_NEW,USER3:USER_NEW..

NUSR=`echo ${UL[*]}|awk -F, '{print NF}'` #finding  number of fields or users
y=1
while [ $y -le $NUSR ] ; do
    USER=`echo ${UL[*]}|awk -F, -v NUSR=$y  '{print $NUSR}' |awk -F: '{print $2}'` #getting     Users to created AKHIL_NEW and SWATHI_NEW and passing to SQLPLUS
    if [[ $USER = SCPO* ]]; then
        TBS=SCPODATA
    else
        if [[ $USER = WWF* ]]; then
            TBS=WWFDATA
        else
            if [[ $USER = STSC* ]]; then
                TBS=SCPODATA
            else
                if [[ $USER = CSM* ]]; then
                    TBS=CSMDATA
                else
                    if [[ $USER = TMM* ]]; then
                        TBS=TMDATA
                    else
                        if [[ $USER = IGP* ]]; then
                        TBS=IGPDATA
                        fi
                    fi
                fi
             fi
        fi
    fi

    sqlplus -s '/ as sysdba'  <<EOF   # CREATING the USERS in the database 
    CREATE USER $USER IDENTIFIED BY $USER  DEFAULT TABLESPACE $TBS TEMPORARY TABLESPACE TEMP QUOTA 0K on SYSTEM QUOTA UNLIMITED ON $TBS;

    GRANT
    CONNECT,
       CREATE TABLE,
       CREATE VIEW,
       CREATE SYNONYM,
       CREATE SEQUENCE,
       CREATE DATABASE LINK,
       RESOURCE,
       SELECT_CATALOG_ROLE
    to $USER;
    EOF 
    y=`expr $y + 1`
done

impdp sysem/manager DIRECTORY=DATA_PUMP DUMPFILE=imp.dp logfile=impdp.log SCHEMAS=AKHIL,SWATHI REMPA_SCHEMA=${UL[*]} 

In the last impdp command I need to get the original users in the dumps i.e AKHIL,SWATHI using the variables.

4
  • 1
    What was the question again?
    – Thor
    Commented Apr 3, 2013 at 12:07
  • 1
    The shell provides elif [[ $USER = WWF* ]]; then to avoid conditions marching off the RHS of the page and scads of fi's. This script (currently) demonstrates why that is a valuable feature. Commented May 11, 2014 at 15:56
  • Possible duplicate of How to use shell variables in awk script
    – tripleee
    Commented Jun 1, 2016 at 15:24
  • also see unix.stackexchange.com/questions/120788/… - using ENVIRON array is preferable to -v due to escaped '\' issues
    – Dani_l
    Commented Sep 25, 2019 at 4:47

6 Answers 6

138

Yes, you can use the shell variables inside awk. There are a bunch of ways of doing it, but my favorite is to define a variable with the -v flag:

$ echo | awk -v my_var=4 '{print "My var is " my_var}'
My var is 4

Just pass the environment variable as a parameter to the -v flag. For example, if you have this variable:

$ VAR=3
$ echo $VAR
3

Use it this way:

$ echo | awk -v env_var="$VAR" '{print "The value of VAR is " env_var}'
The value of VAR is 3

Of course, you can give the same name, but the $ will not be necessary:

$ echo | awk -v VAR="$VAR" '{print "The value of VAR is " VAR}'
The value of VAR is 3

A note about the $ in awk: unlike bash, Perl, PHP etc., it is not part of the variable's name but instead an operator.

8
  • 15
    The answer is NO! You can pass the value of a shell variable to an awk script just like you can pass the value of a shell variable to a C program but you cannot access a shell variable in an awk script any more than you could access a shell variable in a C program. Like C, awk is not shell.
    – Ed Morton
    Commented Apr 3, 2013 at 13:28
  • 4
    @EdMorton You can access environment variables in C. That's what they were originally designed for.
    – Zenexer
    Commented Aug 23, 2016 at 5:56
  • 2
    @EdMorton "The environment" existed before shells that used it came about. Shells were created in part as an easy way to manipulate the environment. See man 7 environ for a very brief overview.
    – Zenexer
    Commented Aug 25, 2016 at 3:08
  • 1
    @EdMorton The goal is to get data from the shell to awk. Using the environment would be an easy way of passing string metadata to a child process at launch; a simple export statement enables this. When shell variables are exported, they become environment variables, but the difference between the two is negligible; even bash treats them identically: 1, 2 I'm not trying to point out a technicality, but rather that you're making things unnecessarily complicated.
    – Zenexer
    Commented Aug 30, 2016 at 21:54
  • 2
    Thank you, @brandizzi. EM is being a bit pedantic methinks.
    – Frobozz
    Commented Nov 15, 2022 at 7:49
33

Awk and Gawk provide the ENVIRON associative array that holds all exported environment variables. So in your awk script you can use ENVIRON["VarName"] to get the value of VarName, provided that VarName has been exported before running awk.

Note ENVIRON is a predefined awk variable NOT a shell environment variable.

Since I don't have enough reputation to comment on the other answers I have to include them here!

The earlier answer showing $ENVIRON is incorrect - that syntax would be expanded by the shell, and probably result in expanding to nothing.

Further earlier comments about C not being able to access environment variable is wrong. Contrary to what is said above, C (and C++) can access environment variables using the getenv("VarName") function. Many other languages provide similar access (e.g., Java: System.getenv(), Python: os.environ, Haskell System.Environment, ...). Note in all cases access to environment variables is read-only, you cannot change an environment variable in a program and get that value back to the calling script.

3
  • 3
    +1 but this is totally incorrect: in all cases access to environment variables is read-only. You can change the environment but only for the current program or programs launched by current program. That said your last line is correct that you can't change environment of current shell by an external program. Another note, shell variable is different from environment variable although for the shell user they appear similar and are accessed in the same way (at least in sh and similar shells). Commented Jul 7, 2016 at 13:12
  • Also see unix.stackexchange.com/questions/120788/…
    – Dani_l
    Commented Sep 25, 2019 at 4:45
  • 1
    Couldn't get this to work, the variable seems never set. TEST=xxx; echo | awk '{print "Test is " ENVIRON["TEST"]}' prints only Test is
    – not2savvy
    Commented Jan 16, 2023 at 11:25
15

There are two ways to pass variables to awk: one way is defining the variable in a command line argument:

$ echo ${UL[*]}|awk -F, -v NUSR=$NUSR '{print $NUSR}'
SWATHI:SWATHI_NEW

Another way is converting the shell variable to an environment variable using export, and reading the environment variable from the ENVIRON array:

$ export NUSR
$ echo ${UL[*]}|awk -F, '{print $ENVIRON["NUSR"]}'
SWATHI:SWATHI_NEW

Update 2016: The OP has comma-separated data and wants to extract an item given its index. The index is in the shell variable NUSR. The value of NUSR is passed to awk, and awk's dollar operator extracts the item.

Note that it would be simpler to declare UL as an array of more than one element, and do the extraction in bash, and take awk out of the equation completely. This however uses 0-based indexing.

UL=(AKHIL:AKHIL_NEW SWATHI:SWATHI_NEW)
NUSR=1
echo ${UL[NUSR]} # prints SWATHI:SWATHI_NEW
6
  • 3
    shouldn't the first one be "print NUSR" instead? Commented Apr 3, 2013 at 12:45
  • My interpretation of the question is that the OP wants the program to do print $2 which prints the second key:value pair, rather than print 2 which prints the number 2.
    – Joni
    Commented Apr 3, 2013 at 12:49
  • 2
    There are more than 2 ways. See question 24 in the comp.unix.shell FAQ at cfajohnson.com/shell/cus-faq-2.html#Q24.
    – Ed Morton
    Commented Apr 3, 2013 at 13:29
  • Hi Joni, your answer was very helpful Commented Apr 4, 2013 at 14:05
  • 1
    $ is not used to access variables in awk; rather, it's an operator. This doesn't do what you think it does. $ is only used to access positional arguments.
    – Zenexer
    Commented Aug 23, 2016 at 6:07
9

There is another way, but it could cause immense confusion:

$ VarName="howdy" ; echo | awk '{print "Just saying '$VarName'"}'
Just saying howdy
$

So you are temporarily exiting the single quote environment (which would normally prevent the shell from interpreting '$') to interpret the variable and then going back into it. It has the virtue of being relatively brief.

4
  • Does this create any potential problems? Seems to be the best way so far
    – Code42
    Commented Apr 30, 2018 at 6:57
  • Well first and most important: It is obtuse. If you were to put it in a shell script it could make for confusing reading, and confusion can lead to mistakes. Second, if you were to put it in a more complicated expression you could find yourself escaping the quotes and $ potentially making the code even more unreadble. Commented May 3, 2018 at 15:28
  • I've used this although it is terrible and will lead to madness if you aren't FREQUENTLY testing your script with something like Bats to ensure quote hell hasn't dragged you down to the 9th level with Dante.
    – dragon788
    Commented Aug 8, 2018 at 21:16
  • @dragon788 Exactly :) Commented Aug 20, 2018 at 16:18
2

Not sure if i understand your question.

But lets say we got a variable number=3 and we want to use it istead of $3, in awk we can do that with the following code

results="100 Mbits/sec 110 Mbits/sec 90 Mbits/sec"
number=3    
speed=$(echo $results | awk '{print '"\$${number}"'}')

so the speed variable will get the value 110.

Hope this helps.

-1

No. You can pass the value of a shell variable to an awk script just like you can pass the value of a shell variable to a C program but you cannot access a shell variable in an awk script any more than you could access a shell variable in a C program. Like C, awk is not shell. See question 24 in the comp.unix.shell FAQ at cfajohnson.com/shell/cus-faq-2.html#Q24.

One way to write your code would be:

UL="AKHIL:AKHIL_NEW,SWATHI:SWATHI_NEW"
NUSR=$(awk -F, -v ul="$UL" 'BEGIN{print gsub(FS,""); exit}')
echo "$NUSR"
echo "$UL" | awk -F, -v nusr="$NUSR" '{print $nusr}' # could have just done print $NF

but since your original starting point:

UL=(AKHIL:AKHIL_NEW,SWATHI:SWATHI_NEW)

was declaring UL as an array with just one entry, you might want to rethink whatever it is you're trying to do as you may have completely the wrong approach.

3
  • 1
    You can access an environment variable from a C program. That's what they were designed for: man7.org/linux/man-pages/man3/getenv.3.html The environment is technically a feature of the OS; shells just happens to utilize it heavily and expose it to the user. The core environment interface is, in fact, only accessible to native (e.g. C) programs. This, of course, all assumes we're talking about POSIX.
    – Zenexer
    Commented Aug 23, 2016 at 5:59
  • Since you know that much I assume you also understand the point I was making and are now just playing the part of the language lawyer (FYI there's an excellent newsgroup comp.lang.c for doing that). Yes, of course you can call a function (getenv in C) or access an array (ENVIRON in awk) to get at the value of a shell variable in a C or awk but what you cannot do is use that shell variable directly in a C program or awk script as you would in a shell script.
    – Ed Morton
    Commented Aug 23, 2016 at 14:47
  • 5
    No, I don't understand the point you're trying to make. Do you mean to say that shell variables can't be accessed directly as variables (i.e., with $ notation)? If so, I don't really think that was the point of the question; that's more of a technicality. As we've seen, yes, it is possible--and quite straightforward--to access the environment from both awk and C.
    – Zenexer
    Commented Aug 25, 2016 at 3:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.