Disclaimer: This is a personal web page. Contents written here do not represent the position of my employer.
Sunday, September 07, 2008
Hackweeks
I'm going to give a brief summary of what I've done in the past weeks (and specially previous week (last one in August), sponsored by Novell as The Hackweek - a form of ITO, in which you can hack at working hours and not only spare time!
I was not so lucky as most of my teammates, who spent the week on the wonderful offices (as they say) of Novell Utah. Maybe if I was in the states already... but well, I had recently an interview in the US Embassy which was the last step of my visa process, so I hope that at least I can make it for the Gnome Boston Summit!
So, firstly I thought of dedicating my week to this task that I submitted to the ideas website, but in the end realized it would be a ton of work to complete without help, and nobody joined me so I decided to learn to use OpenSuseBuildService in order to package Bugzilla, a pretty complex server side software.
Fortunately, I've learnt a lot this week, especially thanks to all the people in irc://irc.freenode.net/#openSUSE-buildservice (above all, darix) and some people from my team (Ray Wang and Stephen Shaw). I could also contact the author of the RPM file for Fedora (who was formerly working for Red Hat), and he's willing to help on joining efforts, as OBS is cross-distribution.
Unfortunately, packaging is a complex task and I didn't finish the package. The documentation is a bit incomplete and having past experience in packaging is a plus that I didn't have. It's awesome that inside our UIA Team we have exclusive resources dedicate to this, because it would be impossible to do from just the developer side.
One of the difficulties I found is finding packages I needed. I could find some of them (BTW, using Webpin is cool for searching on SUSE software repositories, including Packman; except for the fact that doesn't enhance one-click-install) but not all so then I'll have to help on providing packages for some CPAN Perl modules, contributing to the devel:languages:perl official repo.
On the way of learning OBS, I also filed bugs and feature requests (not only to OBS, but also to Banshee! as I have been using it a lot lately at the same time I hack and I even cooked some small patches):
OBS:
Usability issues in project creation page
Link to "My projects" fails if no home:login project has been created yet
New option for uploading the tarball
ChangeLogs parsing is too strict: lines beginning with tab are not recognized, only one date format is accepted
OBS should not allow to create a package named "packageand"
The spec parser should detect the use of a miscplaced packageand(_:_) keyword
Banshee:
No such file or directory errors while importing
Cannot empty some ID3 tag fields
Importing songs without album metadata breaks artist navigation on the iPod
Banshee inserts deduced fictional text on metadata
Should try to locate correct album (and download its cover) in case no album is supplied on metadata
The patches are quite simple so I hope they get committed soon! Clearly Banshee devs and contributors are doing an amazing work, I'm specially amazed by the Moonlight effects, the Muinshee front-end and the effort that it seems is being dedicated to bring Library Sync & Refresh for 1.4. I wish them all the best success.
So let's get back to packaging. Firstly I thought I had found some limitations in OBS or rpm systems because the lack of proper "dual dependency" support. In the past when I installed Bugzilla manually (and updated it to newer versions) I found it a bit hard to do it, specially to deal with DB creation and upgrade. I wanted to make this process easier for the potential rpm user, but without limiting the choice of the DB engine used (as now Bugzilla supports MySQL, PostgreSQL, and Oracle, although the latter was not on my scope). It turns out that it's really hard to apply virtual-provides rules for these cases, as we should stablish more sub-dependencies depending on the db engine you choose, such as perl-DBD-mysql or perl-DBD-Pg, and because it would cause inconsistent situations in case someone in the future replaces his DB-engine with the alternative one (as in theory it should work at the dependencies-level without warning). For references, you may be interested in reading the whole thread about it in the BuildService mailing-list, whose last message includes a possible hack to workaround these problems using patterns (although I don't like to use patterns for solving something like this).
Even discarding the dream about solving this at a package level, it's difficult to solve it at an app level that wouldn't involve reading manuals. My idea was to modify Bugzilla upstream in order to avoid running manually a local script for its initialization, and replace it with a nice web front-end (even if it's only allowed to be run locally). That would cause problems because normally the webserver user doesn't have enough permissions to create files and the database tables and initial data (I'm sure there's always a solution that also doesn't expose security problems, but it's hard to find it as it's noticed in this thread in bugzilla devel newsgroup; so, help is truly welcome).
Maybe the easier and fastest solution is a mixture of both worlds, that is, having a web front-end that asks you the initial configuration, and the form submission process just writes it in a local XML and tells the admin to open a console and run a script as root to finish the process. But even with this solution I guess we should need something to detect at runtime (which should be cross distribution) if some package is installed (which also checks the version). I guess there's already some of that capabilities used in the bugzilla's script checksetup.pl, so I would reuse them. However, in case it finds a problem, there's no way for the application to request installation of a package directly to your OS (but I've attended to a PackageKit conference in last Guadec in which I heard something about this possibility in the future!).
I was not so lucky as most of my teammates, who spent the week on the wonderful offices (as they say) of Novell Utah. Maybe if I was in the states already... but well, I had recently an interview in the US Embassy which was the last step of my visa process, so I hope that at least I can make it for the Gnome Boston Summit!
So, firstly I thought of dedicating my week to this task that I submitted to the ideas website, but in the end realized it would be a ton of work to complete without help, and nobody joined me so I decided to learn to use OpenSuseBuildService in order to package Bugzilla, a pretty complex server side software.
Fortunately, I've learnt a lot this week, especially thanks to all the people in irc://irc.freenode.net/#openSUSE-buildservice (above all, darix) and some people from my team (Ray Wang and Stephen Shaw). I could also contact the author of the RPM file for Fedora (who was formerly working for Red Hat), and he's willing to help on joining efforts, as OBS is cross-distribution.
Unfortunately, packaging is a complex task and I didn't finish the package. The documentation is a bit incomplete and having past experience in packaging is a plus that I didn't have. It's awesome that inside our UIA Team we have exclusive resources dedicate to this, because it would be impossible to do from just the developer side.
One of the difficulties I found is finding packages I needed. I could find some of them (BTW, using Webpin is cool for searching on SUSE software repositories, including Packman; except for the fact that doesn't enhance one-click-install) but not all so then I'll have to help on providing packages for some CPAN Perl modules, contributing to the devel:languages:perl official repo.
On the way of learning OBS, I also filed bugs and feature requests (not only to OBS, but also to Banshee! as I have been using it a lot lately at the same time I hack and I even cooked some small patches):
OBS:
Usability issues in project creation page
Link to "My projects" fails if no home:login project has been created yet
New option for uploading the tarball
ChangeLogs parsing is too strict: lines beginning with tab are not recognized, only one date format is accepted
OBS should not allow to create a package named "packageand"
The spec parser should detect the use of a miscplaced packageand(_:_) keyword
Banshee:
No such file or directory errors while importing
Cannot empty some ID3 tag fields
Importing songs without album metadata breaks artist navigation on the iPod
Banshee inserts deduced fictional text on metadata
Should try to locate correct album (and download its cover) in case no album is supplied on metadata
The patches are quite simple so I hope they get committed soon! Clearly Banshee devs and contributors are doing an amazing work, I'm specially amazed by the Moonlight effects, the Muinshee front-end and the effort that it seems is being dedicated to bring Library Sync & Refresh for 1.4. I wish them all the best success.
So let's get back to packaging. Firstly I thought I had found some limitations in OBS or rpm systems because the lack of proper "dual dependency" support. In the past when I installed Bugzilla manually (and updated it to newer versions) I found it a bit hard to do it, specially to deal with DB creation and upgrade. I wanted to make this process easier for the potential rpm user, but without limiting the choice of the DB engine used (as now Bugzilla supports MySQL, PostgreSQL, and Oracle, although the latter was not on my scope). It turns out that it's really hard to apply virtual-provides rules for these cases, as we should stablish more sub-dependencies depending on the db engine you choose, such as perl-DBD-mysql or perl-DBD-Pg, and because it would cause inconsistent situations in case someone in the future replaces his DB-engine with the alternative one (as in theory it should work at the dependencies-level without warning). For references, you may be interested in reading the whole thread about it in the BuildService mailing-list, whose last message includes a possible hack to workaround these problems using patterns (although I don't like to use patterns for solving something like this).
Even discarding the dream about solving this at a package level, it's difficult to solve it at an app level that wouldn't involve reading manuals. My idea was to modify Bugzilla upstream in order to avoid running manually a local script for its initialization, and replace it with a nice web front-end (even if it's only allowed to be run locally). That would cause problems because normally the webserver user doesn't have enough permissions to create files and the database tables and initial data (I'm sure there's always a solution that also doesn't expose security problems, but it's hard to find it as it's noticed in this thread in bugzilla devel newsgroup; so, help is truly welcome).
Maybe the easier and fastest solution is a mixture of both worlds, that is, having a web front-end that asks you the initial configuration, and the form submission process just writes it in a local XML and tells the admin to open a console and run a script as root to finish the process. But even with this solution I guess we should need something to detect at runtime (which should be cross distribution) if some package is installed (which also checks the version). I guess there's already some of that capabilities used in the bugzilla's script checksetup.pl, so I would reuse them. However, in case it finds a problem, there's no way for the application to request installation of a package directly to your OS (but I've attended to a PackageKit conference in last Guadec in which I heard something about this possibility in the future!).
Labels: CSharp, General, Mono, Mozilla, Programacion, Seguridad, SoftwareLibre, WebDev
# posted by knocte @ 10:22 pm 
Comments:
<< Home
Bugzilla's checksetup.pl already allows you to pass it a config file that contains answers to all the questions it normally prompts the user for. Run ./checksetup.pl --help :)
Also, almost all of the setup/configuration code actually lives in perl modules in the more recent versions of Bugzilla, with checksetup.pl being a user interface wrapper around it. In theory you could have a web UI load those same modules and call into them to do things. You'd just have the permissions issues to deal with (which usually won't be a problem in shared hosting - which is also the usual place you'd actually want a web ui for setup since some don't give you a shell). The file permissions thing could easily be dealt with by setting the permissions appropriately in the package of course, since in most cases the package manager knows how the webserver it depends on is set up.
Post a Comment
Also, almost all of the setup/configuration code actually lives in perl modules in the more recent versions of Bugzilla, with checksetup.pl being a user interface wrapper around it. In theory you could have a web UI load those same modules and call into them to do things. You'd just have the permissions issues to deal with (which usually won't be a problem in shared hosting - which is also the usual place you'd actually want a web ui for setup since some don't give you a shell). The file permissions thing could easily be dealt with by setting the permissions appropriately in the package of course, since in most cases the package manager knows how the webserver it depends on is set up.
<< Home
