21:25:09 <gmcharlt> #startmeeting Dev Meeting, 12 March 2014, part 2
21:25:09 <huginn> Meeting started Wed Mar 12 21:25:09 2014 UTC.  The chair is gmcharlt. Information about MeetBot at http://wiki.debian.org/MeetBot.
21:25:09 <huginn> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
21:25:09 <huginn> The meeting name has been set to 'dev_meeting__12_march_2014__part_2'
21:25:27 <rangi> #info Chris Cormack
21:25:28 <magnuse> yeah, the first meeting was the best ever - with lots of cheese in it ;-)
21:25:40 <mtompset> #info Mark Tompsett
21:25:45 <magnuse> #info Magnus Enger, Oslo Public Library
21:25:50 <mtompset> Don't mention the bonus cheese for those who attend both. ;)
21:26:12 <thd> #info Thomas Dukleth, Agogme, New York City
21:26:23 <gmcharlt> #link http://wiki.koha-community.org/wiki/Developers_IRC_Meeting,_March_12_2014 Agenda
21:26:32 <gmcharlt> #info gmcharlt = Galen Charlton, 3.16 RM
21:27:02 <gmcharlt> #link http://meetings.koha-community.org/2014/koha_dev_meeting__12_march_2014_part1.html Minutes of first part of meeting
21:28:27 <eythian> #info Robin Sheat, Catalyst IT
21:29:03 <gmcharlt> #topic Performance - Plack
21:29:18 <gmcharlt> at the first part there was some discussion about work on Plack at the hackfest
21:29:46 <jcamins> #info Jared Camins-Esakov, C & P Bibliography Services
21:30:01 <gmcharlt> and I believe dpavlin and ashimema will be trying to organize a consolidation of the various threads of Plack support
21:30:23 <gmcharlt> some stuff I know folks are aware of and have patches for are the Zebra reconnection issues
21:31:09 <gmcharlt> I, hopefully notfoolheartedly, announced in-core Plack configs for both intranet and OPAC as a goal for the 3.16 release
21:31:14 <wizzyrea> #info Liz Rea, Catalyst IT
21:31:21 <gmcharlt> comments, particularly from Catalyst?
21:31:41 <eythian> We have one site in production with plack, minor issues have shown up, but nothing world-ending.
21:32:08 <rangi> yeah issues have gone now we have switched to just making a z3950 connection per search and not trying to reuse it
21:32:16 <gmcharlt> OPAC-only or with staff as well
21:32:17 <rangi> (which is what happens under cgi anyway)
21:32:20 <rangi> OPAC only
21:32:34 <eythian> Though, there is a bigger issue when it comes to having many sites on one server using it: that won't work very well atm, as there's no ability to pool resources between them.
21:32:43 <eythian> Solutions to this are possible, I just haven't had a chance to look.
21:32:59 <wizzyrea> even two seems dodgy.
21:33:00 <gmcharlt> OK, well that keeps in the realm of experimental
21:33:06 * magnuse hands eythian a bunch of rund tuits
21:33:27 <gmcharlt> my goal is mostly to have Plack be reasonably functional enough for dev installations that more devs will actually use it that way
21:33:42 <eythian> yeah, there's no reason that won't work pretty well on the OPAC side.
21:33:47 * magnuse thinks that is an excellent plan
21:33:58 <eythian> We're hoping to look into the staff client side some time soon.
21:34:09 <magnuse> eythian: wht resurces would yu want to pool?
21:34:14 <wajasu> i will attempt to signoff 7844 tonight if someone else doesn't beat me to it.
21:34:15 <eythian> magnuse: memory
21:34:30 <mtompset> out of curiousity... if it does get into 3.16 experimentally... does that mean the next release it could go production, or will there be more time than that?
21:34:48 <eythian> magnuse: the current plack system pre-allocates everything, and doesn't "bubble".
21:35:07 <eythian> So, if you have 5 hosts on it, you use 5 times the RAM as when you have one, even when idle
21:35:22 <eythian> so if one site gets busy, there's a whole lot of wasted memory you can't use.
21:35:48 <magnuse> mtompset: no set schedule - it will be ready for production when we think there are no more evil bugs lurking
21:35:53 <magnuse> eythian: ah, thanks
21:36:08 <eythian> (because each site has its own plack instance in order to do UID separation properly.)
21:36:24 * mtompset mutters something about version numbers. ;)
21:36:36 <magnuse> eythian: is that a problem with plack or with koha?
21:36:36 <thd> eythian: is the it to which you are referring Plack or Zebra?
21:36:47 <eythian> magnuse: mostly neither
21:37:24 <eythian> thd: *So, if you have 5 hosts on one server
21:37:28 <eythian> you meant that one?
21:38:01 <jcamins> Just as a comment for anyone who isn't familiar with that part of the code, if C4::Context's context switching were to be resurrected and finetuned, Plack could probably run multiple instances from one process, but you'd lose the user separation, which would be bad.
21:38:08 <eythian> magnuse: if I were to fix it, I'd fix it in plack: allowing it to say "keep 3 around, but go up to 10 if needed and then back down to 3 when thigns are quiet."
21:38:22 <eythian> this may be possible using a different thing than starman.
21:38:41 <magnuse> eythian: sounds like a good thing to have, yes
21:38:54 <eythian> (that is how apache works, fwiw)
21:39:03 <thd> eythian: Yes, multiple hosts per server with an ambiguous 'it'.
21:43:03 <magnuse> eythian: sounds like a problem that needs to be solved before we see widespread adoption of plack among vendors...
21:43:12 <eythian> yup
21:43:13 <magnuse> (in productin)
21:43:30 <eythian> the instance we have running it has its own server for unrelated reasons.
21:43:37 * magnuse looks quizzically at the "o" key on his keyboard
21:44:20 <eythian> and once I did some benchmarking and tuning, making sure it can't run out of ram due to leaks, it's been handling load marvelously.
21:44:43 <gmcharlt> OK, great
21:44:46 <gmcharlt> moving on
21:44:51 <magnuse> but that should not stop us from testing things with plack, using the stuff from bug 7844
21:44:51 <huginn> 04Bug http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7844 enhancement, P5 - low, ---, dpavlin, Needs Signoff , plack scripts for developers
21:44:56 <gmcharlt> #topic Performance and QA
21:45:08 <gmcharlt> #info Question: Should performance be an issue during the QA process?
21:45:24 <gmcharlt> two proposals were made at the first part for policies
21:45:31 <gmcharlt> first, for QA policy -  http://paste.koha-community.org/158
21:45:49 <gmcharlt> second, for an addition to the coding guidelines, http://paste.koha-community.org/160
21:47:01 <gmcharlt> feedback? objections? +1s?
21:47:04 <eythian> That seems reasonable. We do have some bad database latency issues as it is, it'd be good to not have any more.
21:48:05 * magnuse waves to cait
21:48:17 <cait> :)
21:48:23 <cait> the others are still out
21:48:30 <rangi> +1
21:48:50 <mtompset> I voted last time, don't want to double vote. :)
21:48:59 <cait> we had couscous and tajine? tonight
21:49:08 <cait> oh meeting time?
21:49:08 <wahanui> meeting time is always going to favour one section of the globe
21:49:30 <thd> I would like to suggest an amendment.
21:49:34 <wizzyrea> +1 - we should definitely pay close attention to performance in circ.
21:49:49 <cait> #info Katrin Fischer, BSZ Germany
21:50:22 <magnuse> qam in da house!
21:51:08 <gmcharlt> thd: yes?
21:51:49 <thd> Perhaps the policy should acknowledge the possibility that some possible useful features would intrinsically degrade performance and thus may be required to have a system preference to enable or disable them.
21:52:15 <eythian> I'd also like to include database accesses in the coding guidelines, that's where a _large_ proportion of time is spent at the moment.
21:54:17 <gmcharlt> thd: -1 to that amendment -- the text as written is loose enough to cover that possiblity, but I don't think it wise to open the door to either "turn this system preference on to make your Koha catalog slow" or "let's syspref away a bad initial implementation of a feature"
21:56:13 <mtompset> I know in the first meeting there were talks of profiling and perhaps even having some process flag things that generate a 5% slow down.
21:56:15 <gmcharlt> eythian: I don't think database access per se needs to be optimized for -- sure, a lot of the time a good way to avoid latency is to avoid reading from the database when you can read from a faster cache, but that strikes me as an implementation detail, not a matter for a broad guideline
21:56:46 <eythian> gmcharlt: I'd put in the same bucket as bandwidth, storage, and memory.
21:56:50 <gmcharlt> mtompset: yep, there was general agreement that some sort of automated performance testing would be a nice thing to set up
21:58:03 <eythian> gmcharlt: the issue being that currently an opac-search causes 1,718 database hits, which starts to be counted in actual seconds.
21:58:20 <eythian> (for example)
21:58:32 <mtompset> 1718?!
21:59:13 <eythian> http://debian.koha-community.org/~robin/opac-search/ <-- mtompset
21:59:22 <wajasu> N+1 select problem
21:59:36 <gmcharlt> eythian: we don't get billed for database accesses, just time, as it were
22:00:01 <eythian> well, it's specifically a performance issue.
22:00:11 <eythian> and the heading is "Performance" :)
22:00:16 <gmcharlt> I am not disagreeing that a useful strategy would be to reduce the number of queries made, through various means, but I do not see it as a primary thing that we're trying to reduce or conseve
22:00:33 <eythian> fair enough.
22:00:41 <eythian> though it is the biggest single performance killer :)
22:03:03 <gmcharlt> #agreed  http://paste.koha-community.org/158 is accepted as a QA guideline
22:03:22 <gmcharlt> #agreed http://paste.koha-community.org/160 is accepted as a new coding gudeline
22:04:17 <gmcharlt> any new items folks want to discuss?
22:05:02 <eythian> elastic search integration is making progress
22:05:23 <eythian> it works, but it can't yet do anything particularly advanced.
22:05:59 <magnuse> yay for progress
22:06:06 <wajasu> jcamins said i might start testing that stuff.
22:06:51 <eythian> testing is probably a bit of a strong word at the moment, but you're welcome to have a poke at it.
22:08:11 <wajasu> that is a java/lucene based index engine.  so would that be something we would begin to require java jvm, or would it be optional
22:08:33 <eythian> you shouldn't run ES on the same server you're running koha.
22:08:55 <eythian> a production ES installation requires a minimum of 3 servers in a cluster for data integrity.
22:09:12 <eythian> So, you would have your ES servers and point to them from Koha.
22:09:31 <eythian> (I don't expect ES to be a requirement any time in the near future anyway though.)
22:09:32 <magnuse> and zebra should remain as an option as long as there is no z39.50/SRU based on ES, methinks.
22:09:38 <eythian> yup, that too
22:10:43 <gmcharlt> eythian++ # thanks for the update
22:10:48 <gmcharlt> anything else?
22:10:48 <wahanui> anything else is a guess
22:10:56 <gmcharlt> wajasu: forget anything else
22:10:58 <wahanui> gmcharlt: I forgot anything else
22:10:58 <magnuse> in other news, look out for bug 11926
22:10:58 <huginn> 04Bug http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11926 enhancement, P5 - low, ---, wizzyrea, NEW , Render community koha statistic usages
22:11:18 <eythian> ohh
22:11:20 <magnuse> we might try to have it signed off and qa'ed before the end of the week...
22:11:31 <eythian> I want to have realtime stats pushed to logstash one day
22:11:42 <eythian> ah, that's a different thing, ignore me
22:12:01 <wajasu> no. I'm just all in that search code and understand many issues now.
22:12:10 <mtompset> wasn't there discussion of piwik or something on the devel list a while back?
22:13:53 * mtompset goes hunting for the email.
22:14:24 <magnuse> yeah, paul_p talked about having one central piwik install that any koh site could use, but i *think* it was abandoned, because it would be too much data
22:14:50 <rangi> plus its already been done
22:15:03 <rangi> http://piwik.koha-community.org/
22:15:07 <magnuse> there is now a basic dancer app to collect data: https://github.com/clrh/hea-app
22:15:14 <rangi> koha-community.org has been using it for years
22:15:19 <magnuse> rangi: ah, kewl
22:15:23 <rangi> and a couple of other koha
22:16:35 <eythian> can someone who really knows what it is put a description on bug 11926 and the wiki page? It doesn't actually say much about what it is going to do/be.
22:16:35 <mtompset> Isn't just a matter of putting some js in a system preference to use it?
22:16:36 <huginn> 04Bug http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11926 enhancement, P5 - low, ---, wizzyrea, NEW , Render community koha statistic usages
22:16:36 <wajasu> should the schema get regenerated?
22:16:55 <jenkins_koha> Project Koha_master build #1662: SUCCESS in 1 hr 59 min: http://jenkins.koha-community.org/job/Koha_master/1662/
22:16:57 <jenkins_koha> * Fridolin Somers: Bug 11845 - set overlay and import status translatable in addorderiso2709.tt
22:16:58 <jenkins_koha> * Owen Leonard: Bug 10415 - Add course reserves to staff client home page
22:16:58 <huginn> 04Bug http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11845 minor, P5 - low, ---, koha-bugs, Pushed to Master , set overlay and import status translatable in addorderiso2709.tt
22:16:59 <huginn> 04Bug http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=10415 normal, P5 - low, ---, oleonard, Pushed to Master , Add course reserves to staff client home page
22:17:51 <magnuse> eythian: the idea is to enable koha to report stats to the dancer app, like number of records/items, settings for some interesting sysprefs etc
22:18:18 * magnuse has not been involved in the work, but heard a prsentation from the people who have this afternoon
22:21:12 * gmcharlt agrees with eythian that more detail would be nice
22:21:13 <magnuse> the wiki page does sum it up ,i think
22:21:33 <eythian> it doesn't if you have no context.
22:21:40 <wizzyrea> ^^
22:21:55 <wizzyrea> I'm pretty baffled too, tbh.
22:22:44 <magnuse> so, there will be a syspref: do not share data/share data/share data anonymously
22:22:54 <wizzyrea> something like "a central repository for anonymous worldwide koha usage" would be nice, if that's what it is.
22:23:06 <magnuse> if sharing is enabled, every month koha will push data to the dancer app
22:23:15 <magnuse> that's what it is
22:23:32 <magnuse> to be able to know how many libraries use koha, and their sizes
22:24:23 <magnuse> it was also a goal to report on the setting of some sysprefs, to be able to tell how koha is used, if there are features that no-one uses etc
22:24:39 <magnuse> also the versions that are in use
22:24:56 <rangi> but it has to be off by default
22:25:07 <magnuse> yup, of course
22:25:08 * mtompset agrees with rangi. "Off by default."
22:25:09 <rangi> so why would someone go turn it on?
22:25:13 <rangi> or even know too?
22:25:16 <magnuse> and an option to shar anonymously
22:25:31 <mtompset> Perhaps something in the about?
22:25:49 <magnuse> for the same reason they register on the wiki or in web-lib-cats?
22:25:50 <rangi> this feels like something that should be a plugin
22:25:59 <wizzyrea> ^^
22:26:01 <eythian> it seems that it would be better emailing, properly configured servers shouldn't be allowing port 80 access to the outside world anyway.
22:26:09 <rangi> and that
22:26:26 <magnuse> um, a plugin woul hardly increase use?
22:26:41 <magnuse> cool, add your thoughts to the bug or the wiki page folks!
22:26:43 <magnuse> :-)
22:27:42 <rangi> naw i dont think i will
22:27:54 <magnuse> rangi: remember bug 6293? ;-)
22:27:54 <huginn> 04Bug http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=6293 enhancement, P5 - low, ---, paul.poulain, RESOLVED DUPLICATE, Add a button to the intranet for registering with the Koha community
22:28:37 <rangi> yep totally different thing
22:28:40 <wizzyrea> it doesn't continuously collect data
22:28:54 <rangi> a button that takes you off to a form to fill in somewhere
22:28:57 <wizzyrea> and it's just a button, "report in"
22:29:43 <rangi> plus it was a dumb idea then too
22:29:48 <rangi> i have plenty of those
22:30:10 <magnuse> lol
22:30:46 <magnuse> ok, i'll shut up now
22:31:04 <wizzyrea> :)
22:31:23 <gmcharlt> any other topics (briefly?)
22:32:41 <gmcharlt> ok, thanks everybody
22:32:43 <gmcharlt> #endmeeting