21:34:19 #startmeeting Development IRC meeting 26 August 2015 - part 2 21:34:19 Meeting started Wed Aug 26 21:34:19 2015 UTC. The chair is pianohacker. Information about MeetBot at http://wiki.debian.org/MeetBot. 21:34:19 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 21:34:19 The meeting name has been set to 'development_irc_meeting_26_august_2015___part_2' 21:34:41 #topic Introductions 21:34:41 #info wahanui, a bot that has become sentient 21:34:51 Please introduce yourself with #info 21:35:17 #info Jesse Weaver, ByWater Solutions 21:35:35 #info barton, bws, Louisville Ky, USA 21:35:43 #info Tomas Cohen Arazi, Theke Solutions 21:36:50 will leave introductions open until :40 21:36:57 great 21:37:00 eythian? 21:37:00 go back to bed, eythian 21:37:11 it must be beer o'clock 21:38:22 #info Liz Rea, Catalyst IT 21:38:28 tcohen: it's always beer o'clock for wahanui. 21:39:27 cait or cdickinson? 21:40:08 not really here 21:40:10 kk 21:40:15 sorry 21:40:20 no worries 21:41:01 moving on 21:41:08 #info brendan gallagher bywater 21:41:27 #topic Summary from part 1 21:41:41 #link http://meetings.koha-community.org/2015/development_irc_meeting_26_august_2015___part_1.2015-08-26-14.14.html 21:42:17 Things of note: 21:42:28 Plack integration in 3.22 21:42:45 Possibility of adding Koha::Logger to coding guidelines 21:43:22 Work on REST API to simplify auth and integrate into packages 21:43:54 anything else I should mention? 21:44:32 i asked everyone to test the plack integration for the packages 21:44:55 and promised to send a pull request for kohadevbox so it makes use of the new scripts and we can test this 21:45:10 excellent 21:45:15 yes 21:45:21 yep, good stuff 21:46:02 that's it from me 21:46:13 tcohen: would you like to add any RM comments? 21:46:42 i forgot to mention that using plack the way I proposed poses some challenges while trying to solve another ones :-D 21:46:54 such as? 21:47:03 #topic RM 3.22 comments 21:47:23 as i said yesterday here, the apache docs lie about what is supported in which version 21:47:56 so running this as-is requires backporting apache from 14.10 on ubuntu 14.04, and enabling the backports component of Debian 7 21:48:10 Ah 21:48:18 for ubuntu there is a ppa doing it, and for debian it just works adding wheezy-backports 21:48:39 we have enough time to test this, and fix it however we find more siutable 21:48:42 is there no other way to connect to plack? running it directly with mod-fcgid, for instance? 21:49:19 the current approach relied on finding free TCP socket ports for running each plack process 21:49:38 #info Current proposed method for integrating Plack into packages requires a version of Apache only present in Ubuntu 14.10 and wheezy-backports 21:49:44 i shortcircuited that issue, by using Unix Domain sockets instead, taking advantage of apache 2.4's capabilities 21:50:06 pianohacker: i would add that Debian 8 works out of the box 21:50:11 so it sounds less bad :-D 21:50:20 that's Jessie, right? 21:50:29 rigt 21:50:43 i'll talk about this on the list 21:50:54 #info addition to above: also included in Debian 8 (Jessie) 21:51:03 these things sound fixable 21:51:06 yup 21:51:10 or roundaboutable 21:51:18 * wizzyrea is not sure that's a word 21:51:27 round abou table? 21:51:30 :) 21:51:49 wizzyrea: if you don't have the right apache version, it just doesn't use plack, no breakage 21:51:54 (at least :-D) 21:52:22 i have to add (rangi reminded me) that this approach is also suitable for using with nginx 21:52:22 yep, that's what I was picking up, thanks :) 21:52:33 which is yet another integration challenge 21:52:51 anyway, lets move on :-D 21:52:59 zoom. 21:53:02 And that's not really a blocker, IMO; you only get easy plack integration if you have the latest distro version or do a tiny bit of extra work 21:53:15 tcohen: no other comments? 21:53:54 and we'll be moving towards better support, not away, over time. 21:53:59 nope, and that's pretty much all my participation, gotta pick manuel 21:54:00 bye! 21:54:01 #topic RESTful API Implementation 21:54:05 bye tcohen 21:54:09 bye 21:54:14 wizzyrea: agreed 21:55:00 Two main bugs: 21:55:15 http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13799 21:55:16 04Bug 13799: new feature, P5 - low, ---, julian.maurice, Needs Signoff , Add base for building RESTful API 21:55:17 and 21:55:27 http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13920 21:55:27 04Bug 13920: new feature, P5 - low, ---, julian.maurice, RESOLVED DUPLICATE, API authentication system - proposal 21:56:06 regarding this api - rangi asked me to convey the following (he's away from his desk) : I much prefer a lighter approach like Koha::Service that pianohacker has been doing, or the svc/ system However if there is a way to do it without needing an Auth rewrite and a new daemon running, I could live with it. 21:56:47 so I think he's not totally on board with the current approach 21:56:49 as far as moving that proposal forward, the strategy from the earlier meeting was to simplify the authentication system so that it could move forward without a rewrite for now 21:57:04 wizzyrea: agreed, and I think there are a lot of reservations about its complexity 21:57:37 one big question I have is whether this is at least testable without requiring a separate server for mojo 21:58:08 I am all for simplifying - the harder we make it, the slower we will move forward - becuase people are too scared to test, try, activate... 21:58:23 I think this mornings things adressed those wizzyrea - simple auth - no rewrite and tcohen said he’d help with the techinical integration 21:58:52 I'd like to propose that we add a CGI script based on http://search.cpan.org/~mramberg/Mojolicious-4.60/lib/Mojo/Server/CGI.pm 21:58:57 at /api 21:58:58 a challenge on the daemon part - but I think everyone felt like it could be possible 21:58:59 simple auth sounds good - but i still have no idea how to set it up 21:59:09 basically pianohacker's question probably 21:59:24 It would be hidden by an apache rewrite if the daemon is set up, but would work (if slowly) otherwise 21:59:27 thoughts? 21:59:57 pianohacker: can you explain a bit more? 22:00:07 it sounds good, but not sure i understand :) 22:00:10 ^ 22:00:39 my idea is that we create a CGI script literally called api (without a .pl, like the scripts in svc) 22:00:57 that launches the API using Mojo::Server::CGI 22:01:22 it would have to set up everything on every single request, but would allow far easier testing 22:01:44 (this is all on the assumption that the paths for the api are under /api/ ; is that true?) 22:03:12 pianohacker: easier testing sounds good - what would you not need to set up in that case? 22:03:37 the mojo daemon or the apache reconfiguration 22:04:04 ah yeah it would fire one up for each request, and not be daemonised I think I understand. 22:04:08 ^ 22:04:29 so a compromise, having both ways with advantages/disadvantages 22:04:32 that would be poky, but testable 22:04:59 and honestly, my initial thought is also that it would be poky, but we should test and see how terrible it is 22:06:01 because heck, if it's only as slow as any other CGI script, we could just say that to speed up the REST API, set up plack :) 22:06:53 that was a thought that I have heard pianohacker - plack 22:08:15 and plack integration into packages now - that’s nice and simple 22:08:22 #action pianohacker will investigate feasibility of running API as CGI script 22:08:28 if there is urgency on this bug, and I think that there is, making it testable would definitely help. 22:08:42 bag: has anyone run the API as part of a general Plack install? 22:08:43 yes urgency :) 22:08:57 don’t know the answer of yes/no for that question 22:09:10 yeah, we're looking at doing ajax circ using this, and there's, uh, interest in that 22:09:13 (but I don't think that those people who have reservations [smarter people than me] will probably come around to this approach) 22:09:51 that's just your hazard warning. 22:10:16 I’d recommend for anyone/everyone from the above comment - to read the logs from this morning 22:10:20 there was good discussion there 22:10:27 and to speak up :) 22:10:51 I have a massive pile of reservations about this, mostly related to its complexity, the difficulty and boilerplate of adding new API endpoints, testing difficulties, etc 22:11:16 but I think we've got it okay enough that we should try to polish this thing up as much as we can so we don't lose it 22:12:46 any other thoughts about the API for now? 22:13:04 what are the alternatives, if any? 22:13:24 I'm not really counting C4::Service, that's old and silly 22:14:20 There's Koha::Service, which represents the polar opposite approach, but which I don't push much for reasons of conflict of interest: 22:14:23 http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12272 22:14:23 04Bug 12272: enhancement, P5 - low, ---, jweaver, NEW , Refactor C4::Service API into Koha::Service class 22:15:05 that's mostly a cleaner way of doing the old style of service, though 22:16:11 my understanding is that the reasoning behind the new API was starting fresh with a style of API that's a little more internally structured 22:17:07 (I'm still here, just pondering) 22:17:41 What are the absolute most basic things we need out of this? 22:18:03 what is the minimum necessary 22:18:08 api? 22:18:09 rumour has it api is expected to be flexible enough to let us add other search engines later 22:18:12 RESTful, versioned API 22:18:13 wiki? 22:18:13 wiki is http://wiki.koha-community.org 22:19:24 #link http://wiki.koha-community.org/wiki/New_REST_API_RFC 22:20:48 so doing stuff with borrowers, and searching serial items? 22:21:28 that's the thrust of the RFC, though Kyle or I will be shortly adding checkin/checkout 22:21:51 hi 22:21:51 bonjour, eythian 22:22:05 this are the minimums 22:22:22 typing give me a sec 22:22:25 1. Auth 22:22:30 2 get patron status 22:22:41 3. get checkout infromation 22:22:45 4. get hold information 22:22:51 5 get fine information 22:22:56 6 renew checkout 22:23:05 7 create edit delete hold 22:23:12 8 get pickup location 22:24:01 22:24:59 That’s a summary - of course each area has a lot more to them - but that’s the heading :D 22:25:17 Is there an actual specification document anywhere? 22:25:20 public 22:25:35 I thought it was on the wiki that’s why I first said wiki 22:25:39 :D 22:25:46 wizzyrea: I think the above is the closest 22:25:50 but yes it’s been passed around 22:25:57 publicly 22:26:21 #link http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=37413 22:26:54 #info see above for an example of adding the necessary objects, API endpoints and tests for a new API piece 22:27:17 also patron creation 22:27:20 righto 22:27:37 because that's already been written, but got blocked waiting for the API. 22:29:29 yes, patron creation would be an important one to add 22:30:01 I added a goggle doc to the wiki link that shows the full thing for you wizzyrea 22:30:03 https://docs.google.com/document/d/1rWP0RAcYmdlukdacFHremN9TmaKQVtkuEaqpv2i8OHw/edit?usp=sharing 22:30:07 yay thanks! 22:30:49 I think we should send something to the list asking for comments/additions and vote on what the v1 api should include 22:31:14 pianohacker: I feel like we decided that at the hackfest 22:31:29 * rangi wanders past 22:31:36 5 min break 22:31:38 bag: some of that is on the wiki link I posted above 22:31:47 cool thanks pianohacker 22:31:52 but it sounds like there are a lot of missing pieces 22:32:36 rangi: if you have a sec, see what I proposed at :00 about mojo-in-irc 22:32:41 you may have comments on the tech side 22:32:44 So, basically we need this asap because EBSCO is wanting it for their discovery layer, ya? 22:32:45 its worth a try 22:33:00 and because of ajax circ 22:33:06 also, we need to know how to hook it into plack, without emulating cgi 22:33:07 next release wizzyrea 22:33:16 (speaking of that, I'm pretty excited about the angular circ) 22:33:31 so for part one - that’s in the works - no auth rewrite 22:33:49 ie, i dont want to be running plack and mojo side by side 22:34:08 part two - that’s were something needs to be creative 22:34:12 http://stefanorodighiero.net/posts/2014-03-30-managing_multiple_mojolicious_app_with_plack.html ? 22:34:13 also i share all the same reservations, this way overcomplicated and way undertestedable 22:35:03 valid :) 22:35:14 the only reason im not flipping out more, is that without the auth rewrite, it doesnt have the propensity to break everything, in the same way that the rushed hourly loans, or first ajax circ did 22:35:15 hoping it’s testable 22:35:32 it will just break itself, hopefully 22:35:39 right, back to angular training i go 22:35:46 bye 22:35:57 let’s see what 1 brings - then talk with tomas about part 2 - he said this morning that he has ideas there 22:36:13 all right, I think we need to see what the stuff from the earlier meeting brings before we can make more decisions 22:36:16 agreed? 22:36:19 what's 1 and 2? 22:36:32 1. no auth rewrite 22:36:37 2 no daemons 22:36:40 :D 22:36:47 #info Plenty of reservations about the REST API functions, but we can't make any decisions yet because work is still in progress 22:37:12 does that work from non-chairs? 22:37:28 yes to what wizzyrea said 22:37:30 ah, yes, https://wiki.debian.org/MeetBot implies so 22:37:39 all right, I'm closing that topic then 22:37:46 maybe add to it pianohacker - we need at least 4 weeks to really test this 22:37:47 https://algonquincollegesocialmedia.files.wordpress.com/2014/04/tumblr_m7gj9qivt51rxdvy7o1_500.gif 22:37:52 so time is a huge matter here 22:38:06 bag: throw on an #info 22:38:29 #info the API will need at least 4 weeks testing 22:38:32 #info we need at least 4 weeks to really test this - so time really inmportant… I can help with funds - if needed 22:38:43 nice typo bag 22:38:44 double coverage! 22:38:49 bag++ 22:39:04 ytpos++ 22:39:23 #topic Big stuff we are working on 22:40:00 ES needs testing 22:40:03 eythian: anything to add about elastic search? 22:40:55 I might do a writeup some time soon on it. But essentially: ES is mostly working, there are a goodly number of rough burrs to file down, and bits of integration etc. 22:41:03 but at its core, it's doing mostly the right things. 22:41:16 the browse side of it is also working. 22:41:19 So. It is not a big thing, but it is a thing - I'm working on tidying up/standardising some of the interfaces, mostly in my own time. Just alerting you all that it is happening and that you'll see some patches every now and then doing that. 22:41:44 staff interface 22:41:45 staff interface is different, of coruse 22:41:54 I am talking daily with Elastic trying to get them to do training for all the developers - but they can’t get their mind around the fact that I can’t get all the koha developers into the same room 22:42:06 heh 22:42:12 i though they did webinars 22:42:17 you just need a biiiiiig room, bag 22:42:43 I should think they do webinars eythian 22:43:27 https://www.elastic.co/webinars/get-started-with-elasticsearch <-- in fact, they do 22:45:01 I even met with them in person at oscon eythian and they still were not able to answer my questions 22:45:40 ah right. Keep nagging them, I'm sure they'll get the idea :) 22:45:55 At some stage I should write up a big thing saying how I'm using it all, too. 22:47:17 that would be cool - I could hand that to them - and say - hey guys - I’d like a custom webinar that helps us with this… 22:47:30 eythian: mind if I add a #action? 22:47:41 pianohacker: if you like :) 22:48:07 #action eythian to make a writeup showing how he's using ElasticSearch 22:48:17 https://twitter.com/ZacharyTong < here's the guy who does the webinars #imhelping 22:49:14 #imhelping isn't an IRC channel, wizzyrea 22:49:19 zip it 22:49:21 :D 22:49:30 it is now because you clicked it. 22:50:32 unrelated tidbit: I have not been able to convince freshmen in CS that #include is not pronounced "hashtag include" 22:50:48 ... 22:50:49 I 22:50:51 .. 22:50:54 ug 22:50:57 pianohacker: try this one —> it’s prounouced me-me - not meme 22:50:58 aren't guns legal there? 22:51:01 hahahaha 22:51:37 as a teacher, I strongly disapprove of that idea 22:51:59 strongly 22:52:18 ok right 22:52:40 anyway. Not much on the bywater front. Kyle and I will be sending an RFC in the coming weeks to work on an overhaul of the circ rules interface and frontend 22:53:14 to allow for setting rules piecemeal as opposed to the current supertable that requires setting every value for every situation 22:53:15 what are the problems there you are hoping to solve? 22:53:36 wizzyrea: The current system has the following issues: 22:53:50 1) The current interface, with the superwide table, is intimidating and confusing 22:54:19 2) the frontend and backend require that you set every value for every branch/itemtype/categorycode combination 22:54:57 3) The fallback order from specific branch/itemtype/categorycode all the way back to default branch/itemtype/categorycode is confusing as hell 22:55:54 I wrote that verbose list at the top of smart-rules.tt, and I don't remember it! 22:56:02 :) 22:56:29 we have ideas for a new UI and database structure that will make things a lot clearer and more flexible for librarians 22:56:48 and how to do this overhaul piecemeal and not as one huge bug 22:57:22 this is not something that's happening now, but it's on the radar and we're looking for feedback. The RFC will include UI and DB mockups 22:57:24 yeah, I was just wondering how you would split that up into testable bits 22:58:01 gotta run 22:58:04 bye bag 22:58:05 later bag 22:58:15 bye bag 22:58:45 #info ElasticSearch is moving forward, the basics are working. Just needs some refining 22:59:10 #info ByWater have a circ rules revamp on their roadmap 22:59:17 thank you, was just writing that :) 22:59:35 all right! Anything else? 23:00:02 I don't have anything 23:00:25 oh, um, 3.18 is coming along, but there are a few 3.18 only bugs that need poking 23:00:44 wizzyrea: topic 3.18 RM comments? 23:00:57 pianohacker: she's furiously looking things up 23:01:11 I'll take that as a yes :) 23:01:24 #topic 3.18 RMaint comments 23:01:40 pianohacker: btw, make sure your students call it "octothorpe include" for the future. 23:02:07 oh, well, it's not specifically 3.18 but we have a couple of bugs that are waiting on bug 13618 - and they really ought to have interim solutions 23:02:08 04Bug http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13618 normal, P5 - low, ---, jonathan.druart, Needs Signoff , Prevent XSS in the Staff Client and the OPAC 23:02:25 bug 14691 and bug 14505 23:02:27 04Bug http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14691 enhancement, P5 - low, ---, koha-bugs, Needs Signoff , Can't delete patron with ' character in cardnumber 23:02:28 04Bug http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14505 normal, P5 - low, ---, j.kylmala, Needs Signoff , single quotes in journal number cause print routing list window to not appear 23:02:32 ha! That's my course coordinator's preferred pronunciation as well 23:02:39 but we can't decide on a solution 23:02:56 please put your comments on the bug(s) if you have a preferred solution. 23:03:31 #link http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13618 23:03:32 04Bug 13618: normal, P5 - low, ---, jonathan.druart, Needs Signoff , Prevent XSS in the Staff Client and the OPAC 23:03:52 #info the above is preventing progress on the following bugs; please comment with any ideas 23:03:58 #link http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14691 23:03:59 04Bug 14691: enhancement, P5 - low, ---, koha-bugs, Needs Signoff , Can't delete patron with ' character in cardnumber 23:04:04 #link http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14505 23:04:05 04Bug 14505: normal, P5 - low, ---, j.kylmala, Needs Signoff , single quotes in journal number cause print routing list window to not appear 23:04:06 that concludes my concerns I think. 23:04:22 all righty. 23:04:26 #topic GBSD 23:04:34 its pretty vitally important, more important than pretty much everything else that we get that bug 13618 into 3.22 23:04:54 yeah, that bug I'd really like to see tested. 23:05:03 it will solve and prevent so many future problems. 23:05:04 rangi: worth marking it as a blocker? 23:05:17 its a blocker to not being hacked yes 23:05:22 :) well yes 23:05:24 it needs testing i think 23:05:30 all the testing. so much testing 23:05:33 it will break lots of things 23:05:38 maybe smeting for a dedicated sandbox/gbsd? 23:05:39 there is still time to fix most of them though 23:05:50 I'd say that's an excellent segue into the GBSD 23:06:18 #info Bug 13618 would be a great one for global bug squashing day, it is vitally important 23:06:36 #info Reminder, there is a Global Bug Squashing Day on Thursday, September 3 23:06:43 #link http://wiki.koha-community.org/wiki/2015-09-03_Global_bug_squashing_day 23:06:58 #info Please add any bugs/topics of interest to that page 23:07:06 thx pianohacker++ 23:08:05 ... and also. please everyone test the new professional cataloguing editor 23:08:10 please! 23:08:30 I'm going to be fixing the most recent set of feedback ASAP, things have been crazy here at bws 23:08:43 we already got some nice feedback there - but need a little more 23:09:04 I think you really want librarians looking at it eh 23:09:09 #action pianohacker to post google doc to allow per-test-plan-item feedback for Rancor (bug 11559) 23:09:10 04Bug http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11559 enhancement, P5 - low, ---, jweaver, In Discussion , Professional cataloger's interface 23:09:23 do you have a test instance set up that we can point librarians at? 23:09:27 wizzyrea: there is a demo installation for everyone to poke :) 23:09:32 excellent 23:09:36 yup, it's on the bug and the end of the GBSD page 23:09:41 sweet 23:10:02 http://staff-bz11559.bwsdev.bywatersolutions.com/ (login bywater / bywater) 23:10:07 test it test it test iiiiiit 23:10:22 ^ channelling bag since he's gone 23:11:55 I think that's it. Tomas posted in part 1 that he'd post on koha-devel to figure out the time of the next meeting 23:12:00 anything I've forgotten? 23:12:55 naw 23:13:55 givin' til :15 23:16:01 #endmeeting