Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

when and how 'rewriting_compile' function is called? #4

Open
leveryd opened this issue Feb 18, 2021 · 3 comments
Open

when and how 'rewriting_compile' function is called? #4

leveryd opened this issue Feb 18, 2021 · 3 comments

Comments

@leveryd
Copy link

leveryd commented Feb 18, 2021

i read "rewriter.py" source code, "builtins.compile" confused me

@functools.wraps(original_compile)
    def rewriting_compile(*args, **kwargs):
        ...
builtins.compile = rewriting_compile

what i understand at first is "compile function has been hooked, when run 'python xxxx.py' and compile the code, builtins.compile will be called first, so rewriting_compile function will be called".

but after i run test code as below, i am confused.

import dis
import functools
import random
import os
import _frozen_importlib_external
import builtins
from sys import version_info
from ast import PyCF_ONLY_AST

original_compile = builtins.compile

@functools.wraps(original_compile)
def rewriting_compile(*args, **kwargs):
    flags = (len(args) >= 4 and args[3]) or kwargs.get("flags") or 0
    print("rewriting_compile")
    return False

builtins.compile = rewriting_compile

print("end")

"rewriting_compile" will not be printed.

so my question is " when and how 'rewriting_compile' function is called".

@risicle
Copy link
Owner

risicle commented Feb 18, 2021

I'm not sure why it would be called in your case - in python, the compilation step is generally only done when you import a new module (or e.g. use eval to run an arbitrary statement). This is why I mention that install_rewriter() needs to be called very early on in the python process, preferably before any interesting modules are imported.

@leveryd
Copy link
Author

leveryd commented Feb 28, 2021

import functools
import builtins



original_compile = builtins.compile


@functools.wraps(original_compile)
def rewriting_compile(*args, **kwargs):
    flags = (len(args) >= 4 and args[3]) or kwargs.get("flags") or 0
    print("rewriting_compile")
    return False

builtins.compile = rewriting_compile
compile = rewriting_compile


import hashlib

eval("1==1")

# code = compile("", "name", "exec")

if "eval" or "import a new module" will call compile function,why above code does not print "rewriting_compile"?

@risicle
Copy link
Owner

risicle commented Feb 28, 2021

I'm not sure about the eval case (I've never really dug into the inner workings of eval) but for the import hashlib, this is a case that won't call builtins.compile because the bytecode for that module will have already been compiled & written to a .pyc file. Instead it will call _frozen_importlib_external._compile_bytecode when it tries to load that off disk. That's why I have to hook both of them in rewriter.py.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants