Python Decorators and Dynamic Properties

Hello all, and welcome back to my blog,

My progress on MyORM is going kind of slow right now. I have been working on it for a couple of months. This ORM I am making is a redo of my Phase 3 project for FlatIron School which can be found on my Github account (https://github.com/ckelly38/python-p3-v2-final-project-cli/tree/main) (here is the link in case you all want to check it out as my previous blogs probably have you really curious by now, I will say this that not all of this is on there and I did a lot of stuff before I started using GIT). This ORM will be some kind of previous version of SQLAlchemy, but only a bit more flexible and not using Alembic at all. Also I am not compiling the SQL, but I do generate it. Some of the source code for SQLAlchemy believe it or not can be guessed at, as the classes that you as the user built all seem to be run inside of an __init__(self, *args, *kwargs) constructor which seems to be how they get away with doing that kind of stuff. This is also the reason why you cannot set a default value to be used in some sort of validator before an object is created strangely enough.

VALIDATOR METHODS IN SQLALCHEMY:

Speaking of validator and validation methods, I have noticed that SQLAlchemy uses this method, a Python decorator, called validates where you pass in the string for the column name that you are validating. In MyORM, I have a validates decorator method too that I use to register the validator method. Then in MyORM the validator can be looked up by class name, and what columns it validates when it needs to be called, like when you are setting the value. Below is for MyORM, but what SQLAlchemy uses is very similar:

#implementation is not shown for the method in my validator,
#but it checks to see if val is a number of number type
#if not it throws an error
#otherwise if it is a number,
#it checks to see if it is in the given range or not and
#returns that boolean value
@validates("time")
def isvalidtime(self, key, val):
    return myvalidator.isValueInRangeWithMaxAndMin(val, 0, 23);

You will also notice one such difference between my method and SQLAlchemy validator methods. SQLAlchemy validator/validation methods requires you to return the value or throw/raise a(n) value error while my validators methods in MyORM requires you to return a boolean if the value is valid or not, and or may raise/throw an error if some unexpected or illegal input was received. But this makes it easier because the return type is consistent. Although one may argue that the return type of the other was consistently an object, a primitive is not necessarily an object and may have some sort of boxing/unboxing performance problem in some languages. This may also strip the object of some properties needing casting to occur too.

I also talk about SQLAlchemy and constraints and validations in my Phase 4 Blog (https://medium.com/@christopher.kelly1997/phase-4-fis-projects-the-curse-of-using-sql-alchemy-in-your-flask-server-5018de1dfe27) starting with the section called: “DATABASES AND SQL ALCHEMY AND ALEMBIC:” if you want to read about that. If you wanted to read about constraints and validations using SQLAlchemy, you can skip to the section called: “COLUMN AND TABLE CONSTRAINTS AND VALIDATIONS:” and read that.

PYTHON DECORATOR METHODS:

The above is how you call a decorator method with the @ syntax then the method name and the parameters. The function is passed in implicitly, but you must include all parameters needed for both functions.

I have found a really great textbook guide on Python Decorators here (https://www.datacamp.com/tutorial/decorators-python). This article gives like every example imaginable here and is well worth the read.

The main points to take away are:

1. A decorator itself is a function
2. It is a first class function meaning that it takes in a function as an argument/parameter.
3. The decorator returns a function that is defined inside it and customized based on some of the parameters given to it.
4. The decorator often calls the function that came into it and often returns its result inside of the inside-function.
5. Sometimes the inside-function needs to be wrapped with functools to preserve meta information like the function name for example. If this information is essential OR if you have multiple decorators being applied to the function consider using this.
6. You can use an @ character to call a decorator function, but you do not have to. It is a normal function so it can be called like one.
7. You can have many decorators on a function, but they are applied from bottom (closest to the actual function definition) to top (farthest from it) order.

The order of the parameters and which has the function and which has the other parameters or who has a combination of what varies and is not set.

*args are for required positional arguments, and

**kwargs are for keyword arguments.

def decfuncname(*args, **kwargs):
   def contfuncname(myfunc):
       #do something with the function like call it for example
       #perhaps get the result and then do something with it before returning it.
       #This area really is up to you.
       result = myfunc(args, kwargs);
       return result;
   return contfuncname;

What I actually used for my validates method:

def validates(*args):#cls, 
    #print();
    #print(f"cls = {cls}");
    #print(f"keys = {args}");

    def myAddVDator(myvfunc):
        #print(f"funcname = {myvfunc.__name__}");
        #print(dir(myvfunc));
        #print(dir(myvfunc.__code__));
        #print(f"funcclassfullname = myvfunc.__qualname__ = {myvfunc.__qualname__}");
        #print(f"keys = {args}");

        mkys = [];
        for ky in args:
           if (type(ky) in [list, tuple]):
               if (len(args) == 1):
                   for item in ky:
                       if (type(item) == str): mkys.append(item);
                       else: raise TypeError("the type of item on keys must be a string!");
               else: raise ValueError("args must only have the list of keys on it!");
           elif (type(ky) == str): mkys.append(ky);
           else: raise TypeError("each key must be a list a tuple or a string");
        #print(f"mkys = {mkys}");
            
        #the code for this will be different if using under version 3 of Python.
        mcnm = myvfunc.__qualname__[0: myvfunc.__qualname__.index(myvfunc.__name__) - 1];
        #print(f"mcnm = {mcnm}");
        #print();

        mycol.addValidator(mcnm, myvfunc.__name__, mkys);
            
        #because this is a decorator and not actually calling the function we need to return it
        #this also prevents getattr seeing the function name and thinking that it is None.
        return myvfunc;
    return myAddVDator;

Some things to note about this validates method and why it is different. I call addValidator method inside the containing function. But that has no return value. So I cannot return it. I discovered here that in this case I actually needed to return the function that was the input because I did not actually call it.

Not doing so resulted in me having a defined function in the file, but when I went to get it by name using getattr on the class, the function returned None due to returning None in the validator, so I in fact needed to return the function here. Yes that bug was fun to figure out what happened.

I did extract meta information from the function as that was more what I was interested in; like the function name and the containing class (IE what Python 3 calls the full qualified function name; the difference between that and name is that the full qualified name also has the containing class in front of it.the method name). I also did some conversion from tuple to list and some validation of the keys information here. As you can see just like in SQLAlchemy my keys argument can be a list or a string or multiple. So that means I am allowing multiple column validator methods. The single column validator methods will always be called before the multi-column validators in MyORM. I will also note here that I am compiling to Python version 3 so that one line I have noted is dependent for that version. For other versions, that line in particular is different. See https://stackoverflow.com/questions/961048/get-class-that-defined-method for more information on that line. Now I am going to talk about Python Properties.

PYTHON PROPERTIES AND THE PROPERTY DECORATOR METHOD:

Image of property setter and getter decorator and dynamic important lines only no getter and setters here though.

Python also provides us with a property decorator. Properties are very useful in Python because they control how a(n) class attribute is accessed. That is nice, but if you are new to using them, you might be a little confused. So I am going to break it down for you.

#inside of class whatever:

def getItem(self): return self._item;

def setItem(self, val):
    #the set code often checks to see if the value is valid first if it can
    #but eventually the setter does when valid:
    #if it is not valid it throws a ValueError(“value for ___ not valid!”);
    self._item = val;

item = property(getItem, setItem);
#NOTE: the property decorator has other stuff too like delete item and has item, and a default value if not found, and a description of it, or you can choose not to provide these

The example above is without a decorator. This creates a property for whatever the class was and the property is called “item”. To access it in the get, we use “_item”. The _ is so there is no recursion error.

Without the _, this is infinitely recursive based on how the getattr works, because it tries to get item, item tells it to use the getItem method, but when it tries to get the item it sees that it needs to use the getItem method, therefore it never ends without that underscore.

Also the _ denotes a private attribute here in Python. That is also why you need the _ in the set too. So you don’t accidentally overwrite the property and cause access hell. We call the one with the _ in front of the name the private name. The one without is the public name.

“item” public name

“_item” private name

One other thing to note about the property method that you do not see here, technically the property decorator applies the access restrictions to a class attribute that is accessed via an object. Strangely enough, that is the best way to explain it. I will do a better attempt at explaining this later on.

So keeping in mind the above example, let us look at it with the property decorator method now:

#inside of class whatever:

@property
def item(self): return self._item;

@property.setter
def item(self, val): 
    #the set code often checks to see if the value is valid first if it can
    #but eventually the setter does when valid:
    #if it is not valid it throws a ValueError(“value for ___ not valid!”);
    self._item = val;

Note you call it the name of the property that you want and include the required parameters. I prefer the first way, but up to you. You will see why I prefer the first way. So the above is all fine and dandy, but what happens when you want to declare properties dynamically and still want the access controls, but you want to do this at runtime?

DYNAMIC PROPERTIES:

I will first note that this has been discussed and it seems that there is no consensus on what you actually need to do this. No one way I found is consistent and there seem to be a couple of different approaches on how to do this.

These links were not that helpful, but did provide some insight into the problem (you may find these more useful or perhaps exactly what you need):

https://stackoverflow.com/questions/60686572/dynamic-property-getter-and-setter

https://stackoverflow.com/questions/10967551/how-do-i-dynamically-create-properties-in-python

Get Christopher Hugh Kelly’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

https://stackoverflow.com/questions/37869030/how-to-programmatically-define-properties-in-python

There are a few things to keep in mind about properties:

• remember in Python properties have the normal property name and the private name
• (when I say public or normal name, think “item”;
• when I say private name think “_item”).
• remember to get and set from/to the private name to prevent the infinite loop

Now keeping those things in mind, now it gets interesting.

THE GETTER AND SETTER:

Your getters and setters will actually return a function. They should be similar to what I have below:

def myObjectGet(self, attrnm):
    def fget(self):
        #print("INSIDE GET!");
        #print(f"attrnm = {attrnm}");
        myattval = getattr(self, "_" + attrnm);
        #myattval = __getattr__(self, attrnm);
        #print(f"myattval = {myattval}");
        return myattval;
    return fget;

def myObjectSet(self, attrnm):
    def fset(self, val):
        #print("INSIDE SET!");
        #print(f"attrnm = {attrnm}");
        #print(f"val = {val}");
        myattval = getattr(self, attrnm);
        myattval = val;
        #myattval.value = val;
        setattr(self, "_" + attrnm, val);
        #print("DONE WITH SET!");
    return fset;

The inner get method (fget) calls getattr(self, “_” + attrnm); and returns this value.

The outer get method (myObjectGet) returns the fget function customized by the given attribute name to use.

The inner set method (fset) calls getattr(self, attrnm); for the public attribute name and then sets it to the new value. Then we also call setattr(self, “_” + attrnm, val); which sets the private attribute to the new value. There is no return in the inner set and that is ok.

The outer set method (myObjectSet) returns the fset function customized by the given attribute name to use.

THE DYNAMIC SETTING:

So those are the getter and setter methods that you need for the new properties. Now what? We need to actually create the properties. Often this is created from some sort of data. Then of course the data is iterated through and the properties are generated in some sort of loop.

for n in range(len(mobjnms)):
    mobj = mobjs[n];
    mnm = mobjnms[n];
    print(f"mobj = {mobj}");
    print(f"mnm = {mnm}");
            
    #setattr(self, mnm, mobj);
    setattr(type(self), mnm, property(self.myObjectGet(mnm), self.myObjectSet(mnm)));
    setattr(self, "_" + mnm, mobj);
            
    #properties need a get, set functions as well as a name
    #properties are applied to the class only not just the self object...
    #but you cannot only do it on self, otherwise the access methods will not be enforced.
    #properties have a certain name like camper
    #but to access it you need to do self._camper ...
    #
    #so first you need to create the property
    #then you need to set the private value
    #now you can try getting the value, but the getter needs to pull it from the private value
    #then you can officially attempt to set the value using the property
    #this will then tell the setter to get the current value of the property and say it is val
    #then set the private value
            
            
    #you do not actually need these lines these are to verify that it works
    print("val of the new attribute should be the initial value!");
    print(getattr(self, mnm));
    #print(getattr(self, mnm).fget(self));
            
    print("now setting the value here.");
    #getattr(self, mnm).fset(self, mobj);
    setattr(self, mnm, mobj);
    #you do not actually need these lines these are to verify that it works
    print("val of the property should now be the object!");
    print(getattr(self, mnm));

THE IMPORTANT LINES FROM ABOVE:

The important lines in the loop above are:

setattr(type(self), mnm, property(self.myObjectGet(mnm), self.myObjectSet(mnm)));
setattr(self, "_" + mnm, mobj);
setattr(self, mnm, mobj);

And to verify that it works you would access it with:

print(getattr(self, mnm));

Note: from above the “mnm” is the attribute name I want to use.

The “mobj” is the new value I want to use for it. This may be None (or null in other languages), but it may also be defined.

THE ALL IMPORTANT CREATION LINE:

The all important first line in creating the dynamic property:

setattr(type(self), mnm, property(self.myObjectGet(mnm), self.myObjectSet(mnm)));

First thing to notice about this dynamic property is that you need to create it as a property as the value.

But this needs to be applied to the class and not the individual objects to ensure that the access methods are working properly. You may have noticed the: #print(“INSIDE GET!”); and #print(“INSIDE SET!”); That is what those lines were for. To ensure that the access methods were working properly.

What the first setattr line does is it takes the object that needs to have the value applied to it (that is the class (IE: type(self) type of self)).

Then it takes in the attribute name what you want to call the new attribute and what you want to use to access it with (“mnm” variable).

Now for the value. This is the property call itself:

property(self.myObjectGet(mnm), self.myObjectSet(mnm))

You will notice that “myObjectGet” and “myObjectSet” methods got passed in, but they also got called with the attribute name (“mnm”) here. Remember these methods return a function the “fget” and “fset” which are used for the getter and setter methods. Those were the methods from above.

Finally, these methods needed to come from somewhere, you will recall that the “fget” and “fset” need the object (“self”) that holds the data. That is what the “self.” in front of those methods is there for.

THE OTHER TWO LINES SETTING THE VALUE:

The other two important lines:

setattr(self, "_" + mnm, mobj);
setattr(self, mnm, mobj);

I made two other setattr calls. They both had “mobj” which is actually the new value I wanted to include there. The “self” object of course will be holding the newly added dynamic property, so make sure to call it with the class you actually wanted and not necessarily the class where the method is in, but it must extend it.

• In this case, “mymodel” class (whatever the user wanted to call their model class) extends “mybase” class.
• “mybase” class holds the method that does the dynamic setting.
• However “mymodel” class has an instance which is the “self” object. This is what is housing the dynamic properties.
• In this case, it is important to note that the “self” object must be an instance of a subclass of “mybase”, but not “mybase” class.

First you will notice, I set the private name of this property first. That is so the get method does not fail if I want to check the value. That is important.

Second you will notice, I set the public name of this property. Now the values are the same, but the “fset” method got called in the process.

Thanks for reading everyone, I hope you found this helpful and really useful. Hopefully, I will see you again next time. Dated: 4–21–2025.