Access an installed libray

franz-program · September 9, 2021, 10:58am

Hello, I was wondering if there may be a way to access and modify the libraries (its .py files) installed from ‘python_packages’ under configuration of the add-on

xaviml · September 9, 2021, 8:45pm

Hi @franz-program,

It could be done with init_commands, but if you need to change many things, it could be a nightmare. Could you maybe explain the reason why you want to do this? Maybe your problem has another solution than modifying libraries’ py files.

Regards,
Xavi M.

franz-program · September 10, 2021, 6:33am

One of the reasons I use AppDaemon is to automatically change my config files without writing them on my own, but the library I use to write (ruamel) behaves rather strangely when writing quotes (it’s a common asked question on StackOverflow).

Therefore I looked into that and luckily I found two lines of code where the creator adds quotes when they’re not needed (I know, it’s bad practice to change a library).
I already asked how to override this function on StackOverflow (since it’s not in the Main class it’s not that easy to manage) but no luck so far.

Anyway, how can I use init_commands to modify the library? (I’m ready to take this risk) Thanks for the answer though

xaviml · September 11, 2021, 2:34pm

Hi @franz-program,

My first recommendation for you is that you report the bug you mention in GitHub, so it can be fixed from the root source.

However, if you are still insisting in changing the code’s libraries, then you could do it either from the terminal addond (connecting through docker to appdaemon container), or through the init_commands from the configuration. The latter is more difficult, but more “stable”, because if you restart AppDaemon, anything changed in the container, you will lose it.

Since you didn’t give me the detail of the common asked question on StackOverflow, I don’t know which lines you want to change, but this is an example of how to change the version number of the library for example:

system_packages: []
python_packages:
  - ruamel.yaml==0.17.16
init_commands:
  - sed -i 's/0.17.16/0.99.99/g' /usr/lib/python3.9/site-packages/ruamel/yaml/__init__.py
  - "python3 -c 'from ruamel.yaml import __version__; print(f\"The ruamel modified version is: {__version__}\")'"

The python3 command is not needed, it is just to show in the console the modified version. The key here is the sed command, which is replacing all the 0.17.16 to 0.99.99 from the /usr/lib/python3.9/site-packages/ruamel/yaml/__init__.py file.

Bear in mind that if the file change its location, or the line changes, this might need to be modified again. So I recommend you pinning the ruamel version, so the changes are always consistent.

Hope this helps achieve what you need.

Regards,
Xavi M.

franz-program · September 13, 2021, 7:47am

The two lines are more of a ‘creator choice’ rather than a bug, but it’s pretty annoying regardless.

Anyway really thanks for the answer, I looked the ‘sed functions’ up on the internet and I found out (it’s actually new for me) what it can do.
I tried it by writing this on the configuration:

system_packages: []
python_packages:
  - ruamel.yaml==0.17.14
init_commands:
  - >-
    sed -i 's/return '"'/return ''/2'
    /usr/lib/python3.9/site-packages/ruamel/yaml/emitter.py
  - >-
    sed -i 's/return "'"/return ''/g'
    /usr/lib/python3.9/site-packages/ruamel/yaml/emitter.py

//this format is actually done automatically I didn't write it 
//this way

But I get fatal error on the log:

/var/run/s6/etc/cont-init.d/appdaemon.sh: eval: line 54: unexpected EOF while looking for matching `"'
/var/run/s6/etc/cont-init.d/appdaemon.sh: eval: line 55: syntax error: unexpected end of file

I thought maybe it had to do with the quotes therefore I put:

return \'\"\'/return \'\'

but I get the same.

I wonder why it does say looking for matching ` " ’ . (also the first is not the single quote but its ASCII CODE is 96) Is it ‘forgotting’ the return?

Anyway huge thanks for your answer.

xaviml · September 13, 2021, 8:03am

Hi @franz-program,

Could you maybe link in here the lines of the file you want to change? (GitHub link preferrably) And let me know what is what you want to replace? It will help me understand what do you want to achieve.

Regards,
Xavi M.

franz-program · September 13, 2021, 9:21am

Yeah of course. In this line: I’d like to remove the or (so that it is true only if \n has to be dumped in a yaml documnet).
In this other line: I’d like to return an empty string and not the quotes.

Just to explain you why I’d want this: like I said I use appdaemon to automatically change my yaml files. In one of them I have to print (or dump as it’s said) ‘on’ but so far it didn’t work since the libraries actually dumps " ‘on’ " (HAAS doesn’t quite like it). I searched how to impose quotes on yaml files and also the creator said to change some lines (other lines actually, not the one I’d like to change).
Of course I tested the changes on another compiler and it worked so far (that doesn’t mean it’s not gonna create any problem perhaps).

xaviml · September 13, 2021, 9:02pm

Hi @franz-program,

This works for me:

system_packages: []
python_packages:
  - ruamel.yaml==0.17.14
init_commands:
  - sed -n 873p /usr/lib/python3.9/site-packages/ruamel/yaml/emitter.py
  - sed -n 879p /usr/lib/python3.9/site-packages/ruamel/yaml/emitter.py
  - sed -i 's/"'\''" in self.event.value or //g' /usr/lib/python3.9/site-packages/ruamel/yaml/emitter.py
  - sed -i 's/return "'\''"/return ""/g' /usr/lib/python3.9/site-packages/ruamel/yaml/emitter.py
  - sed -n 873p /usr/lib/python3.9/site-packages/ruamel/yaml/emitter.py
  - sed -n 879p /usr/lib/python3.9/site-packages/ruamel/yaml/emitter.py

The first 2 and the last 2 are there to show in the console that it worked. This is what is shows to me:

if "'" in self.event.value or '\n' in self.event.value:
   return "'"
if '\n' in self.event.value:
   return ""

In case that this does not work. Another solution that just crossed my mind is that you could fork the project on GitHub, change the lines you want, and then add the git url to the python packages. That should work as well. For example, I installed ruamel like this:

system_packages: [git]
python_packages:
  - git+https://github.com/alvistack/ruamel-yaml-code.git
init_commands: []

You will just need to point to your repository. This solution should be more stable since the code will be pulled everytime you start the container, and the version will not change unless you change it.

Let me know if you have any questions

Regards,
Xavi M.

franz-program · September 14, 2021, 10:19am

yeah, It does work and now I can print quotes however I want. Huge huge thanks!!

franz-program · September 21, 2021, 8:28am

if I may ask, since I’m trying to use the second way you said (with github), how can I do it? I guess it’s a rather obvious question but I’m no github expert unfortunately. Thanks

xaviml · September 24, 2021, 8:47am

Hi @franz-program,

These are the steps to follow:

Login with GitHub
Go to https://github.com/alvistack/ruamel-yaml-code
Click in the fork button. This will create a copy of the code in your GitHub profile, something like: https://github.com/YOUR_USERNAME/ruamel-yaml-code
Change the lines that you want to have. You can do this in different ways:
- Cloning the project locally
- Pressing “.” in your GitHub project and edit it from there
- Editing it directly with the pencil button ign GitHub
Now you can change your AppDaemon addon configuration to the following and replace YOUR_USERNAME for your username in GitHub:

system_packages: [git]
python_packages:
  - git+https://github.com/YOUR_USERNAME/ruamel-yaml-code.git
init_commands: []

Hope this gives you an overall idea of how to do it. I could do it if you will in my account, but then you will rely on me not to delete this project in the future for any unknown reason. So it is better that you do it in your GitHub profile.

Regards,
Xavi M.