3

NB: I am using Lua version 5.3.


This question is motivated by Exercise 25.1 (p. 264) of Programming in Lua (4th ed.). That exercise reads as follows:

Exercise 25.1: Adapt getvarvalue (Listing 25.1) to work with different coroutines (like the functions from the debug library).

The function getvarvalue that the exercise refers to is copied verbatim below.

-- Listing 25.1 (p. 256) of *Programming in Lua* (4th ed.)

function getvarvalue (name, level, isenv)
  local value
  local found = false

  level = (level or 1) + 1

  -- try local variables
  for i = 1, math.huge do
    local n, v = debug.getlocal(level, i)
    if not n then break end
    if n == name then
      value = v
      found = true
    end
  end
  if found then return "local", value end

  -- try non-local variables
  local func = debug.getinfo(level, "f").func
  for i = 1, math.huge do
    local n, v = debug.getupvalue(func, i)
    if not n then break end
    if n == name then return "upvalue", v end
  end

  if isenv then return "noenv" end   -- avoid loop

  -- not found; get value from the environment
  local _, env = getvarvalue("_ENV", level, true)
  if env then
    return "global", env[name]
  else        -- no _ENV available
    return "noenv"
  end

end

Below is my enhanced version of this function, which implements the additional functionality specified in the exercise. This version accepts an optional thread parameter, expected to be a coroutine. The only differences between this enhanced version and the original getvarvalue are:

  1. the handling of the additional optional thread parameter;
  2. the special setting of the level parameter depending on whether the thread parameter is the same as the running coroutine or not; and
  3. the passing of the thread parameter in the calls to debug.getlocal and debug.getinfo, and in the recursive call.

(I have marked these differences in the source code through numbered comments.)

function getvarvalue_enhanced (thread, name, level, isenv)
  -- 1
  if type(thread) ~= "thread" then
    -- (thread, name,  level, isenv)
    -- (name,   level, isenv)
    isenv = level
    level = name
    name = thread
    thread = coroutine.running()
  end

  local value
  local found = false

  -- 2
  level = level or 1
  if thread == coroutine.running() then
    level = level + 1
  end

  -- try local variables
  for i = 1, math.huge do
    local n, v = debug.getlocal(thread, level, i) -- 3
    if not n then break end
    if n == name then
      value = v
      found = true
    end
  end
  if found then return "local", value end

  -- try non-local variables
  local func = debug.getinfo(thread, level, "f").func  -- 3
  for i = 1, math.huge do
    local n, v = debug.getupvalue(func, i)
    if not n then break end
    if n == name then return "upvalue", v end
  end

  if isenv then return "noenv" end   -- avoid loop

  -- not found; get value from the environment
  local _, env = getvarvalue_enhanced(thread, "_ENV", level, true)  -- 3
  if env then
    return "global", env[name]
  else
    return "noenv"
  end

end

This function works reasonably well, but I have found one strange situation1 where it fails. The function make_nasty below generates a coroutine for which getvarvalue_enhanced fails to find an _ENV variable; i.e. it returns "noenv". (The function that serves as the basis for nasty is the closure closure_B, which in turn invokes the closure closure_A. It is closure_A that then yields.)

function make_nasty ()
  local function closure_A () coroutine.yield() end
  local function closure_B ()
    closure_A()
  end

  local thread = coroutine.create(closure_B)
  coroutine.resume(thread)
  return thread
end

nasty = make_nasty()
print(getvarvalue_enhanced(nasty, "_ENV", 2))
-- noenv

In contrast, the almost identical function make_nice produces a coroutine for which getvarvalue_enhanced succeeds in finding an _ENV variable.

function make_nice ()
  local function closure_A () coroutine.yield() end
  local function closure_B ()
    local _ = one_very_much_non_existent_global_variable  -- only difference!
    closure_A()
  end

  local thread = coroutine.create(closure_B)
  coroutine.resume(thread)
  return thread
end

nice = make_nice()
print(getvarvalue_enhanced(nice, "_ENV", 2))
-- upvalue  table: 0x558a2633c930

The only difference between make_nasty and make_nice is that, in the latter, the closure closure_B references a non-existent global variable (and does nothing with it).

Q: How can I modify getvarvalue_enhanced so that it is able to locate _ENV for nasty, the way it does for nice?


EDIT: changed the names of the closures within make_nasty and make_nice.


EDIT2: the wording of Exercise 25.3 (same page) may be relevant here (my emphasis):

Exercise 25.3: Write a version of getvarvalue (Listing 25.1) that returns a table with all variables that are visible at the calling function. (The returned table should not include environmental variables; instead it should inherit them from the original environment.)

This question implies that there should be a way to get at the variables that are merely visible from a function, whether the function uses them or not. Such variables would certainly include _ENV. (The author is one of Lua's creators, so he knows what he's talking about.)


1 I am sure that someone with a better understanding of what is going on in this example will be able to come up with a less convoluted way to elicit the same behavior. The example I present here is the most minimal form I can come up with of the situation I found by accident.

3
  • 1
    I'll say this to be safe: don't use code like that in production. It breaks as soon as you precompile code without debug symbols. it should only ever be used for ad-hoc debugging or optional debugging components. Jul 23, 2019 at 11:41
  • @DarkWiiPlayer: thank you for the advice. To be clear, which code are you referring to? All the code in the post?
    – kjo
    Jul 23, 2019 at 12:04
  • 2
    In general, don't rely on debug, but more precisely, everything that relies on variable names completely breaks if you strip debug symbols when compiling with luac. It's only really viable when you're in control of what code you work with, so not at all when writing a library to publish. Jul 23, 2019 at 13:46

2 Answers 2

1
  local function inner_closure () coroutine.yield() end
  local function outer_closure ()
    inner_closure()
  end

The function make_nasty below generates a coroutine for which getvarvalue_enhanced fails to find an _ENV variable; i.e. it returns "noenv"

That's a correct behavior.
The closure outer_closure has upvalue inner_closure but doesn't have upvalue _ENV.
This is how lexical scope works.
It's OK that some closures don't have _ENV upvalue.

In your example the closure inner_closure isn't defined inside the body of outer_closure.
inner_closure is not nested in outer_closure.

2
  • Thank you, but your answer does not address at all the question that I actually asked at the end of my post.
    – kjo
    Jul 23, 2019 at 12:01
  • @kjo - You can explicitly insert the line _ENV=_ENV into outer_closure so that it surely has _ENV upvalue. If you're not able to modify outer_closure then you're unable to get its _ENV because it doesn't have it. But it has an upvalue that does have the upvalue _ENV, so you might try to search the whole tree of dependencies :-) Jul 23, 2019 at 14:07
1

It's impossible.

If a closure doesn't make any use of the global environment _ENV, then it doesn't have that upvalue whatsoever.

A function like

local something = 20

local function noupval(x, y)
   return x * y
end

Doesn't need or have any upvalues, not even to the global environment.


This question implies that there should be a way to get at the variables that are merely visible from a function, whether the function uses them or not.

There really isn't though. You can easily confirm this by looking at the output of luac -p -l <your_code.lua>, more precisely at the upvalues of each function.

If anything, I think using the word visible is somewhat misleading there. Visibility really only matters when creating a closure, but once it has been closed, that closure only has a set of upvalues which it can access.


Exercise 25.3: Write a version of getvarvalue (Listing 25.1) that returns a table with all variables that are visible at the calling function. (The returned table should not include environmental variables; instead it should inherit them from the original environment.)

You may have misunderstood that exercise; the way I understand it is something like this:

local upvalue = 20
local function foo()
  local var = upvalue -- Create 1 local and access 1 upvalue
  type(print) == "function" -- Access _ENV so it becomes an upvalue
  return getvarvalue_enhanced()
end

and the above would return {var = 20, upvalue = 20, _ENV = <Proxy table to _ENV>}

After all, it asks specifically about the calling function, not one you pass as a parameter.

This doesn't change the fact that you still only get _ENV if you access it though. If you don't use any globals, the function won't have any reference to _ENV whatsoever.

5
  • Thank you for these observations. Your noupval example is a bit misleading/confusing, though, because, after all, no upvalues are even defined outside of it, whether it uses it or not. Your point would be clearer (to me at least) if you added a local something = ... before the definition of noupval.
    – kjo
    Jul 24, 2019 at 11:00
  • Please see my EDIT2; it may be relevant to your answer.
    – kjo
    Jul 24, 2019 at 11:42
  • 1
    @kjo I expanded my answer a bit ;) Jul 24, 2019 at 12:18
  • @DarkWiiPlayer - {var = 20, upvalue = 20, _ENV = <Global Environment>} Probably this is not what was asked for. It's not enough to put _ENV in the table. "Inherit env variables" means create a metatable for such inheritance. Jul 25, 2019 at 8:14
  • @EgorSkriptunoff That's true, I seem to have misunderstood that the first time. Either way, implementing that is a completely different proble than what OP asked for, so I'm not going to expand my answer with an explanationof how to do this. Jul 25, 2019 at 8:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.