How I refresh my Zope products

By: Peter Bengtsson, 19th of June 2006

19^th of June 2006

This is for all Zope developers out there who write Python products for their web applications. Some of you restart Zope every time you make a change, some of you click "Refresh this product" in Control_Panel/Products/YourProduct, some of you have have "Auto refresh mode" and some of you will soon have my script instead.

An obvious disadvantage with restarting Zope is that it takes time. It can take several seconds on a site filled with lots of product packages such as CMFPlone. So, you make a change in your code, shift to another window where you have your Zope running; you hit Ctrl-C, up arrow, Enter, wait, shift to the Firefox window and press F5 (alternatively Ctrl-R). That's unacceptably slow. When I complained about Zope 2.9.x breaking producting refreshing on the Zope mailinglist several developers told me to forget about product refreshing and use unittests and Zope restart. That tells me that there are still people who use this inefficient way of doing it. Another problem with restarting Zope is that you loose any session variables you have. Perhaps you use REQUEST.SESSION for login to some application which means that you'll have to restart that process too manually.

For those who still use the "Auto refresh mode" on selected few products you're probably aware of two problems:

1) the refresh isn't actually done until you really need it which leads to sluggish response times

2) dependencies break

Another problem with older Zope versions was that you used to get error messages about "ValueError: Cache values may only be in one cache" sporadically when using REQUEST.SESSION and "Auto refresh mode".

So, how do I do it? It's quite simple actually. I use the product refresh button inside the Control_Panel/Products except that instead of manually mouseclicking I let a script do it for me. The script can run in two "modes". Either with the inotify interface builtin with most recent Linux kernels or a continously looping script that checks for file differences every 5 seconds. When the script discovers a file difference (based on some "clever" filtering rules) it programatically clicks the "Refresh product" button inside the Control_Panel.

Advantages about this script approach:

• instant refresh of product without having to instigate it with a browser refresh
• ability to refresh without restarting Zope
• works even when not in debug-mode
• gives instant feedback about errors in the code (1)
• takes care of dependencies (2)

(1) since I'm running the script constantly in an open shell window, when there's an error the traceback is immediately printed in red until the next save.

(2) When I define which products I want the script to keep an eye on, I also write which other products depends on the one I'm configuring.

This script that I have is called click_on_product_refresh.py is very much inspired by bin/load_site.py for those familiar with it. I give it away as Open Source under the ZPL license and I do confess that its code is a bit messy. At least it works and has been used and tested by yours truely for a long time now. Using it is easy because what you do is that you write a little "configuration script" that defines all the parameters for you can then calls click_on_product_refresh.py with the correct setup. A configuration script can look like this:

 #-- Start of configuration -----------------------------------------------

 libpath="lib/python"
 CTurl="http://localhost:8080/Control_Panel/Products/IssueTrackerProduct"
 productpath="Products/IssueTrackerProduct"

 refresh_type = "clever" # or 'forced'
 netrc_authenticator = "zope"

 #-- End of configuration -------------------------------------------------

 import click_on_product_refresh
 click_on_product_refresh.cli(globals())

I put this file in /home/peterbe/zope/zope285 where click_on_product_refresh.py is also located. The netrc stuff is optional but I recommend that you use it. If you're unfamiliar with .netrc you just have to create a plain text file called ~/.netrc that looks like this:

 machine zope
     login peterbe
     password secretpassword

That's necessary because the click_on_product_refresh.py script effectively needs to log in to Zope every time it does a refresh.

The difference between "clever" and "forced" for the refresh_type variable is that with "clever" it only refreshes on files that actually need refreshing. For example, a change to a .zpt file of a Zope in debug-mode doesn't require a refresh. "forced" makes it refresh on any change to any file; that's useful when you need to refresh products in live production states like when you launch an upgrade in a production environment and you don't want to restart Zope.

If you have multiple products you want to monitor for changes, you can define them all as a list. It can then look like this:

 #!/usr/bin/python

 #-- Start of configuration -----------------------------------------------

 libpath="/usr/lib/zope-285/lib/python"
 products = (
  dict(
    CTurl="http://localhost:8080/Control_Panel/Products/IssueTrackerProduct",
    productpath="Products/IssueTrackerProduct",
    dependencies = ["IssueTrackerFiles", "IssueTrackerStatistics"]
  ),
  dict(
    CTurl="http://localhost:8080/Control_Panel/Products/IssueTrackerFiles",
    productpath="Products/IssueTrackerFiles",
  ),
  dict(
    CTurl="http://localhost:8080/Control_Panel/Products/IssueTrackerStatistics",
    productpath="Products/IssueTrackerStatistics",
  ),
 )

 refresh_type = "clever" # or forced
 netrc_authenticator = "zope"

 #-- End of configuration -------------------------------------------------

 import click_on_product_refresh
 click_on_product_refresh.cli(globals())

Because products IssueTrackerFiles and IssueTrackerStatistics are based on IssueTrackerProduct (ie. base-classing from IssueTrackerProduct) every refresh of IssueTrackerProduct also refreshes IssueTrackerFiles and IssueTrackerStatistics.

Download all of these files into your /my/zope/root and edit the file refresh_example.py at the same time as you rename it to something. (I suggest you rename it to 'refresh_all.py and use the same script for all your Zope products.

I know I could probably describe this even better but consider this a first draft. If time allows I might try to explain it with a screencast or something that shows how it works. If you're into Zope2 product development and don't already have a similar solution I can genuinely garantee that this will speed up your productivity.

Comment