0

I'd appreciate some help with the following code, as I'm still relatively new to Python, and despite countless days trying to figure out where i'm going wrong, i cant seem to spot the error i'm making.

I've adapted the following code from an article on medium to create a logging decorator and then enhanced it to try and "redact pandas df and dictionary" from the logs. Using functools caused me a problem with pytest and pytest fixtures. A post on stack overflow suggested dropping functools in favour of decorators.

def log_decorator(_func=None):
    def log_decorator_info(func):
        def log_decorator_wrapper(*args, **kwargs):
        
            _logger = Logger()
            logger_obj = _logger.get_logger()

            args_passed_in_function = args_excl_df_dict(*args)
            kwargs_passed_in_function = kwargs_excl_df_dict(**kwargs)
           formatted_arguments = join_args_kwargs(args_passed_in_function,kwargs_passed_in_function)
        
            py_file_caller = getframeinfo(stack()[1][0])
            extra_args = { 'func_name_override': func.__name__,'file_name_override': os.path.basename(py_file_caller.filename) }

            """ Before to the function execution, log function details."""
            logger_obj.info(f"Begin function - Arguments: {formatted_arguments}", extra=extra_args)
        
            try:
                """ log return value from the function """
            
                args_returned_from_function = args_excl_df_dict(func(*args))
                kwargs_returned_from_function = []
                formatted_arguments = join_args_kwargs(args_returned_from_function,kwargs_returned_from_function)
            
                logger_obj.info(f"End function - Returned: {formatted_arguments}", extra=extra_args)
       
            except:
                """log exception if occurs in function"""
                error_raised = str(sys.exc_info()[1])
            
                logger_obj.error(f"Exception: {str(sys.exc_info()[1])}",extra=extra_args)
            
                msg_to_send = f"{func.__name__} {error_raised}"
               send_alert(APP_NAME,msg_to_send,'error')
            
               raise
            return func(*args, **kwargs)
        return decorator.decorator(log_decorator_wrapper, func)
    if _func is None:
        return log_decorator_info
    else:
        return log_decorator_info(_func)

Having adapted the above code i cant figure out what is causing the following error

args_returned_from_function = args_excl_df_dict(func(*args)) TypeError: test_me() takes 4 positional arguments but 5 were given

Other functions which the log decorator relies on

def args_excl_df_dict(*args):
    args_list = []
    for a in args:
        if isinstance(a,(pd.DataFrame,dict)):
            a = 'redacted from log'
            args_list.append(repr(a))
        else: 
            args_list.append(repr(a))
    return args_list

def kwargs_excl_df_dict(**kwargs):
    kwargs_list = []
    for k, v in kwargs.items():
        if isinstance(v,(dict,pd.DataFrame)):
            v = 'redacted from log'
            kwargs_list.append(f"{k}={v!r}")
        else:
            kwargs_list.append(f"{k}={v!r}")
    return kwargs_list

def join_args_kwargs(args,kwargs):

    formatted_arguments = ", ".join(args + kwargs)
    return str(formatted_arguments)

This is the code calling the decorator

@log_decorator.log_decorator()
def test_me(a, b, c, d):
    return a, b

test_me(string, number, dictionary, pandas_df)
4
  • Why is the decorator @log_decorator.log_decorator() and not just @log_decorator ? is the first part a class name or module name or namespace?
    – Alexander
    May 9, 2022 at 21:31
  • Its a module name - log_decorator.py
    – Khu
    May 9, 2022 at 21:41
  • and what about decorator.decorator in the return statement? Where does that come from?
    – Alexander
    May 9, 2022 at 23:11
  • This is a module thats imported from PyPI - "import decorator". I could see part of the problem i was having using functools and pytest fixtures, was pytest fixtures werent being retained by the decorator - similar to the issue mentioned on this thread - stackoverflow.com/questions/19614658/…. I therefore tried to follow this example of using decorator to see if that would resolve the problem but got the above instead
    – Khu
    May 10, 2022 at 10:09

2 Answers 2

1

I think the problem is that the wrapper is including the function as an argument to the function.

Try adding this line and see if it helps

args = args[1:]

intor your log_decorator_wrapper function towards the top. Like this.

def log_decorator(_func=None):
    def log_decorator_info(func):
        def log_decorator_wrapper(*args, **kwargs):

            args = args[1:]   # < -------------------here
            _logger = Logger()
            logger_obj = _logger.get_logger()

            args_passed_in_function = args_excl_df_dict(*args)
            kwargs_passed_in_function = kwargs_excl_df_dict(**kwargs)
           formatted_arguments = join_args_kwargs(args_passed_in_function,kwargs_passed_in_function)
        
            py_file_caller = getframeinfo(stack()[1][0])
            extra_args = { 'func_name_override': func.__name__,'file_name_override': os.path.basename(py_file_caller.filename) }

            """ Before to the function execution, log function details."""
            logger_obj.info(f"Begin function - Arguments: {formatted_arguments}", extra=extra_args)
        
            try:
                """ log return value from the function """
            
                args_returned_from_function = args_excl_df_dict(func(*args))
                kwargs_returned_from_function = []
                formatted_arguments = join_args_kwargs(args_returned_from_function,kwargs_returned_from_function)
            
                logger_obj.info(f"End function - Returned: {formatted_arguments}", extra=extra_args)
       
            except:
                """log exception if occurs in function"""
                error_raised = str(sys.exc_info()[1])
            
                logger_obj.error(f"Exception: {str(sys.exc_info()[1])}",extra=extra_args)
            
                msg_to_send = f"{func.__name__} {error_raised}"
               send_alert(APP_NAME,msg_to_send,'error')
            
               raise
            return func(*args, **kwargs)

        return decorator.decorator(log_decorator_wrapper, func)
    if _func is None:
        return log_decorator_info
    else:
        return log_decorator_info(_func)
1
  • 1
    Appreciate the help, looks like it solved the problem. Thanks again @alexpdev
    – Khu
    May 11, 2022 at 22:17
-1

If your code is as is in your editor, maybe look at the indentation on the first three functions. Then start from there to move down

1
  • This was just copy paste error into stack and its properly indented on VSC
    – Khu
    May 9, 2022 at 21:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.