14:01:20 #startmeeting Development IRC meeting 8 August 2018 14:01:20 Meeting started Wed Aug 8 14:01:20 2018 UTC. The chair is Joubu. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:01:20 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 14:01:20 The meeting name has been set to 'development_irc_meeting_8_august_2018' 14:01:22 no wahanui 14:01:28 #chair cait kidclamp 14:01:28 Current chairs: Joubu cait kidclamp 14:01:28 wahanui: ping! 14:01:34 hey! :) 14:01:37 #topic Introductions 14:01:44 #info Jonathan Druart 14:01:44 dev meeting now.. I'll get back to this after 14:01:47 #info Katrin Fischer, BSZ, Germany 14:01:51 #info Owen Leonard, Athens County Public Libraries, USA 14:01:58 #info Martin Renvoize, PTFS Europe, UK 14:02:11 #info Colin Campbell, PTFS-Europe, UK 14:02:15 #info Victor Grousset, BibLibre, France 14:02:27 #link https://wiki.koha-community.org/wiki/Development_IRC_meeting_8_August_2018 14:02:29 #info Andrew Isherwood, PTFS Europe, UK 14:02:40 (refresh it if opened) 14:03:37 kidclamp, tcohen, jajm? 14:03:49 #topic Announcements 14:03:52 Anyone have something? 14:04:02 they were here only a few minutes ago.. I'm sure 14:04:30 well.. not seen kidclamp today actually 14:04:36 nopd 14:04:38 nope 14:04:53 #topic Update from the Release Manager (18.05) 14:05:07 hi 14:05:07 #info RM was in holiday, back this week and will push all the things 14:05:17 #action kidclamp push all the things 14:05:28 (that's why you should attend meetings) 14:05:29 kidclamp++ ;) 14:05:30 #topic Updates from the Release Maintainers 14:05:34 #info Tomas Cohen Arazi 14:05:41 kidclamp++ 14:05:42 ashimema, something? 14:05:43 heh 14:06:13 Been pushing a fair bit this week after a backlog after vacation.. I'm all up to date and awaiting kidclamps next set of pushes 14:06:33 #info 18.05.x: I have been pushing a fair bit this week after a backlog after vacation.. I'm all up to date and awaiting kidclamps next set of pushes 14:07:15 also, I caught a series of bugs that had got missed between handover.. had a good tidy up of bugzilla 14:07:18 I feel like 18.05.02 is finally ready for production 14:07:27 at least to catch the last bugs 14:07:39 * tcohen agrees 14:07:55 ashimema++ 14:07:56 #info If you plan to use 18.05.x in production, pick 18.05.02 or later 14:07:58 yeah.. feels pretty stable at this point.. though there's a fair few more minor bugs already making their way into 18.05.x for 18.05.03 14:08:49 think that's it from me 14:08:50 ashimema++ 14:09:07 ashimema++ 14:09:17 yep ashimema++ good job on following bug fixes :) 14:09:21 #topic Updates from the QA team 14:09:31 I believe fridolin is up to date on 17.11 and 17.05 too.. though he's away imminently and will struggle to keep up this month 14:09:52 cait, something? 14:09:53 sorry.. that was a bit late ;).. over to you QA 14:10:00 waiting for ashimema 14:10:12 the qa queue is in an ok shape 14:10:32 #info qa queue is in an ok shape, slowly going down, below 40 now 14:10:45 :D 14:10:50 problem is, we have a lot of bugs that sit there for quite a time and nothing easy left 14:11:02 * ashimema wonders if we need to promote bug 20773 for opinions 14:11:02 04Bug http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20773 major, P5 - low, ---, koha-bugs, Needs Signoff , bug 20724 follow-up - Database cleanup 14:11:11 quite a lot of elasticsearch bugs 14:11:23 2 criticals and 9 majors waiting for SO 14:11:32 yes, that's the other problem 14:11:43 the only reason we are in good shape in QA is that the sign-offs are way too slow 14:11:49 tons of bugs waiting there 14:11:53 and numbers going up 14:12:12 so we really need sign-offs atm I think 14:12:32 #info Needs Signoff is way too high with lots of bugs waiting for testers - please sign-off! 14:12:47 ashimema: can you explain about the bug above? 14:13:06 Right now, the reason I'm not doing more signoffs is I'm not confident I know how to test properly. I don't know if it should be my job to ask or the patch-writer's job to make it easier. 14:13:18 just passing by to shill for bz8000, there's already 2 SO on it 14:13:28 #info next step for Elasticsearch is Bug 18316 - Add weighting/relevancy options to ElasticSearch. Additional testing/sign offs might make it move faster! 14:13:29 It's a db cleanup script as a followup to bug 20724. I think it's ready for signoff but Joubu commented it needs discussion? 14:13:29 04Bug http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20724 critical, P5 - low, ---, jonathan.druart, Pushed to Stable , ReservesNeedReturns syspref breaks "Holds awaiting pickup" 14:13:51 oleonard: it's good to get feedback on the reasons 14:14:01 can you give some examples to look over? 14:14:17 i think patch writer sshould always include a test plan 14:14:24 seafarmer: Bug 8000 is marked "needs signoff." Is that incorrect? 14:14:24 04Bug http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8000 enhancement, P5 - low, ---, charles.farmer, Needs Signoff , Test mode for notices 14:16:30 sounds like it should be SO from a quick skim read 14:16:35 seafarmer: it only has one sign-off - the sandboxes one was ed himself i tihnk 14:16:42 Sorry cait I don't have an example off the top of my head. 14:16:57 and it had bugs fixed after the sign-off so back to get tested again i think 14:17:13 well sleuthed cait 14:17:56 hm also wondering how it could be tested with sandboxes 14:18:05 does email work there? 14:18:17 not on ptfs ones right now ;) 14:18:37 cait: looking at the message_queue table 14:18:44 #info Thomas Dukleth, Agogme, New York City 14:19:17 moving on? 14:19:24 i'd think so 14:19:30 moving on 14:19:32 #topic General development discussion (trends, ideas, ...) 14:19:41 I have added several ones there 14:19:48 In short, you all should sign off on my patches :P 14:19:54 so quickly, starting with 14:20:03 #topic Bug 13618 - Prevent XSS in the Staff Client and the OPAC 14:20:03 04Bug http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13618 enhancement, P5 - low, ---, jonathan.druart, Needs Signoff , Prevent XSS in the Staff Client and the OPAC 14:20:28 QA team and everybody concerned by security issues (so, all of your, right?) must take a look at that one 14:20:45 I have been working on it for months (years?) now and we have finally a working and ready solution 14:20:54 this is an important one and need to be pushed ASAP 14:21:01 I should sign that one off.. I've commented that it looks solid.. I just wanted to get a branch setup and throw a pen test tool at it first 14:21:13 :) 14:21:13 I personnaly do not care that much, I have written scripts to do the rebase job 14:21:45 have you thrown any pen test tools against it to varify it's caught everything? 14:22:04 so you can wait for 2 more years if you like :) (i.e. that was my last call about it) 14:22:14 everything? lol :) 14:22:19 I'll add it back to my list.. just it's a long list ;) 14:22:27 I wrote a script to insert problematic data in the DB 14:22:36 haha.. well everything an automated pen tool may catch ;) 14:22:43 talking about security, ig to another one 14:22:48 to test that one, you will need time, patience, and understand the issues globally 14:23:16 ashimema: we will have a unit test, to parse the .tt files 14:23:21 indeed 14:23:23 #info input needed on how to best make Bug 15836 - Labels: Offer configuration option for splitting call numbers secure. 14:23:45 all [% %] will need to have a filter (even if not filtered, it will do nothing) 14:24:14 the script I wrote to generate the patch will more or less be what we will need to write the regression tests 14:24:49 cait: that is another topic right? 14:25:15 cait: please add it to the wiki, I am processing the item 1 by 1 14:25:26 #topic Bug 21010 - Add a script to search for data inconsistencies 14:25:26 04Bug http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21010 enhancement, P5 - low, ---, koha-bugs, NEW , Add a script to search for data inconsistencies 14:25:27 OK.. I'm happy to have further discussion with Joubu after this meeting to move 13618 on.. we can move on topic in the meeting I reckon 14:25:35 ashimema: yep! 14:25:48 will do 14:25:57 #info Martin Renvoize to SO 13618 and discuss next steps with Jonathan 14:26:01 I think we really need a script to catch data inconsistencies (at DB level so) 14:26:22 I have started with authority types, item types and items.$branch 14:26:31 done 14:26:45 the idea is to provide a script that will catch problematic situations 14:26:46 :+1: 14:26:55 Joubu: Should it be a built-in report which can be accessed from the staff client? 14:26:59 then we could move it to the UI if needed 14:27:07 :) 14:27:09 oleonard: ^ :) 14:27:22 I think it would be very useful! 14:27:29 if you think about something else, just open a new bug report and link it to that one 14:27:39 would be nice for it to run with any/all upgrades and set a flag in about 14:27:48 much like some of the other warnings we have in there 14:27:54 refresh wiki if you want - all bugs linked now too 14:28:04 if you have ideas, please comment on bug 21010 :) 14:28:48 moving on in few sec 14:28:52 I presume that 21010 is intended for some fundamental DB inconsistencies. 14:29:00 #info if you have ideas for data consistencies to look out for, file a bug and link it to bug 21010 14:29:35 #topic Bug 20443 - Move C4::Members::Attributes to Koha namespace 14:29:35 04Bug http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20443 enhancement, P5 - low, ---, jonathan.druart, In Discussion , Move C4::Members::Attributes to Koha namespace 14:29:36 Now I understand better, a meta-bug. 14:29:59 yes, see linked bug reports 14:30:18 okies 14:30:30 wwhat about it 14:30:38 bug 20443 is quite a big refactoring one, it adds several interesting things to the Koha:: namespace 14:30:44 41 files changed, 713 insertions(+), 2017 deletions(-) 14:30:53 Joubu: that's too much 14:31:08 ^ that's the interesting bit, we remove 2/3 of the code 14:31:22 tcohen: ? 14:31:35 it touches too many things, that's it 14:31:45 there are 10 commits 14:31:53 it's not a monolitic commit 14:32:08 when I move the code I describe what I am doing, commiting the things 1 by 1 14:32:19 we'll see :-P 14:32:22 it's easier to have only 1 bug report and test everything 14:32:34 * ashimema is working through reviewing it.. but it is indeed a pretty massive piece.. 14:32:44 than have 10 bug reports and have to test all the features 10 times 14:32:45 Joubu, could it be split into 10 bugs then ? testing/QAing that kind of bug can be really long 14:32:56 jajm: what I just said 14:33:22 It's safer as it. Otherwise I need to provide 10 stable states 14:33:29 and wait for 10 testers and 10 QAers 14:33:48 In that case it is in effect a monolithic commit 14:34:05 oleonard: for testers it's easier 14:34:05 okay, but it's easier to miss something in a 3000 lines long diff 14:34:09 I was more thinking perhaps a couple of QA team could get together to review it as a set for half a day.. bit of peer coding/reviewing. 14:34:10 we need to coordinate this testing I think 14:34:16 for QAer I have split the commits, to make them easy to read 14:34:18 +1 14:34:26 2 QA members take on it together 14:34:39 i like the idea 14:34:42 jajm: read the log, it's not a 3000 lines diff ;) 14:34:42 to make it move faster 14:34:48 I volunteer 14:34:51 tcohen.. feel like you've got the time/inclination to take it on with me at some point? 14:34:53 would it help to schedule a qa day again? 14:35:04 ashimema: absolutely 14:35:04 I can try and dedicate a monday or tuesday afternoon (to overlap with your morning?) 14:35:07 but also can be outside, just asking :) 14:35:13 ashimema++ 14:35:18 tcohen++ too 14:35:37 done.. we'll hit it together.. will grab you out of meeting to schedule 14:35:45 moving on, there are 5 more 14:35:55 #topic Bug 18252 - Move C4::Items code to the Koha namespace 14:35:55 04Bug http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18252 enhancement, P5 - low, ---, jonathan.druart, ASSIGNED , Move C4::Items code to the Koha namespace 14:36:10 #info Martin and Tomas are going to schedule a paired QA for bug 20443 14:36:10 04Bug http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20443 enhancement, P5 - low, ---, jonathan.druart, In Discussion , Move C4::Members::Attributes to Koha namespace 14:36:10 #action ashimea and tcohen to review bug 20443 together 14:36:17 ah typo... 14:36:24 but info is good 14:36:31 its ok 14:36:47 I am going to submit the same kind of changes for C4::Items, I have local patches to replace GetItem with Koha::Items->find, like we did for Koha::Patron the idea is to use Koha::Item objects everywhere and simplify the code 14:36:58 at least for CRUD in a first step 14:37:11 sounds good to me 14:37:17 but it should be done really soon 14:37:29 we had some side effects last time, so should have some time to hunt them down 14:37:37 and then I will open a discussion after the meeting about that patches :) 14:37:59 maybe if done before kohacon, we could try and get people on it there 14:38:03 yes, side effects are expected with that changes, they usually highlight issues with the current code. So it's good 14:38:08 (not for the end user) 14:38:29 #info Bug 15836 - Labels: Offer configuration option for splitting call numbers 14:38:29 04Bug http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15836 enhancement, P5 - low, ---, jonathan.druart, Needs Signoff , Labels: Offer configuration option for splitting call numbers 14:38:42 cait: ^ 14:39:04 Joubu has written this very nice feature, but we are a little stuck on the best way to handle the regex in a secure way 14:39:36 so it would be cool if people could take a look 14:39:47 i think it's a really cool feature for libraries not using DDC/LOC 14:40:03 because at the moment only those break nicely on multiple lines on labels 14:40:37 i already #info'd earlier, so leaving it at that now if there are no questions 14:40:42 cait: can you explain the problem? 14:40:48 So basically the issue is: we using stored regexs to split on the callnumber 14:40:56 handing over to joubu ;) 14:40:58 like we store "s/foo/bar/g" 14:41:18 of even better: "s/(foo)bar/$1pouet/g" 14:41:49 btw... example on the bug is a real life example from the sponsoring libraries 14:42:01 to evaluate it, we will need to use eval { $callnumber =~ $regex }; 14:42:10 something like that, cannot remember the exact syntax 14:42:57 if the $regex contain something else than a regex, perl will evaluate it anyway (rm, ls, etc.) 14:43:07 read the commit message for more info ;) 14:43:36 I have described several solutions, I do not think they are acceptable 14:44:09 we need to know how we can trust on data in the DB 14:44:58 tcohen told me that we already have this problem: letter.content can contain TT directives and so execute whatever commands 14:45:21 ouch. 14:45:35 [% PERL %] is allowed ? 14:45:43 everything is allowed 14:46:11 double ouch 14:46:20 maybe we should move that discussion out of th emeeting 14:46:25 Is there no standard Perl module for limiting such input? 14:46:28 like into a security bug 14:46:28 yes, moving on 14:46:43 #info ILS-DI GetRecords (see ML) 14:47:17 working on removing GetItem (using in ILS-DI subs) I realize that it's not clear to me what this service is supposed to return 14:47:50 we have an example in the opac/ilsdi.pl view, but for instance GetRecords does not have anything in the tag 14:47:58 whereas it returns a lot of stuffs 14:48:22 the ils-di is supposed to allow you to link Koha to a discovery interface (DI) 14:48:27 git log told me it's historical and should not be there, but... who knows? 14:48:28 and i don't see any use there for issues 14:48:48 actually, i'd rather say it's unnecessary and could be a privacy issue 14:49:28 ok, it can be an answer for that one, but for the other info? 14:49:49 can I contact someone using it heavily to know which are the info they need? 14:50:03 the specs are not clear 14:50:08 what is the other info? 14:50:14 I think this is a shortcut someone took 14:50:20 and not part of the spec 14:50:26 whose using it heavily thouhg ;) 14:50:34 biblibre 14:50:43 they use a Drupal <-> Koha connector 14:50:50 cait: if we add columns in some tables, it will be added to the xml 14:50:51 that at some point was implemented on this I think 14:51:04 but I think they moved to koha-restful 14:51:20 we could check if the vufind driver uses it for something 14:51:28 that might cover some other cases 14:51:34 I would vote to push it closer to the spec if there was a vote 14:51:36 tcohen, the koha connector still use some parts of ils-di 14:51:58 but yeah.. would be worth askin users 14:52:45 "A list of record objects that represents the bibliographic record, as well as 14:52:45 associated holdings and item information for each requested bibliographic 14:52:45 identifier (unless the specified return schema does not support it). If a hash is 14:52:46 returned, the bibliographic identifier would be the key." 14:52:53 https://old.diglib.org/architectures/ilsdi/DLF_ILS_Discovery_1.0.pdf 14:53:18 ok, moving on, will see later if I need more info 14:53:22 ILS-DI allows basically anything to be returned 14:53:38 fair enough 14:53:59 i think record and items 14:54:05 from what you cited 14:54:05 tcohen: «biblibre they use a Drupal <-> Koha connector» 14:54:06 And not only. Also Bokeh (http://bokeh-library-portal.org) uses it. 14:54:14 reading that for a second or third time it does look more open to opinion than I thought it did first time 14:54:53 record, holdings and item information.. does 'issues' constitute item information is the question I suppose 14:54:59 tcohen: «but I think they moved to koha-restful» For some things, ILS-DI is still use IIRC 14:55:14 I read issues as 'historical loan information' when i read the first email.. is that right 14:55:15 is koha-restful still a thing? 14:56:01 cait: yes https://git.biblibre.com/biblibre/koha-restful 14:56:23 ashimema, i think the question should be "do we want backward compatibility ?" and i think the answer is yes and we should not remove anything for the ils-di responses 14:56:25 We still have to maintain it 14:57:04 With hope that the official API will make it useless someday 14:57:14 jajm: seen the callnumberX for the last 3 checkouts in the answer? 14:57:22 interesting, thx tuxayo 14:57:38 Joubu, no 14:58:08 it comes from a commit of 2005 that added a sub, then this sub has been used for ILS-DI GetRecords 14:58:16 i still feel checkout history doesn't belong there 14:58:33 and we ended up with these fieldsX (not callnumber actually, cardnumber!) 14:58:45 ok moving on, 2 minutes to go 14:58:52 #info Last projects to move to gitlab/k-c (should contrib be split?) 14:58:57 ashimema: for you 14:59:16 :) 14:59:18 there are some projects not moved to gitlab yet: release-tools, contrib and maybe more 14:59:43 release-tools: nobody really uses it, so I did not 15:00:00 but can be done, maybe just a mirror of the git.k-c.org 15:00:06 sandboxes? 15:00:16 sandbox is used, but maybe it could be extracted into its own repo ? 15:00:18 ho, I actually created 15:00:19 https://gitlab.com/koha-community/release-tools 15:00:26 that's the one 15:00:29 1 sec :) 15:00:31 I think splitting out sandboxes could be useful indeed 15:00:34 ah yes, it makes sense, we could split up the contrib one 15:00:39 I use release-tools now ;) 15:00:40 +1 for moving sandboxes 15:00:46 and submitted fixes too recently ;) 15:00:52 the problem with release-tools and contrib is: we have it on git.k-c.org and what about issues, patches, sync? 15:01:02 indeed 15:01:20 I imagine a cleanup could happen on contrib.. 15:01:21 either we remove them from git.k-c.org, or we need to think and setup something that fit our needs 15:01:36 is koha-plack on there ever used now for example 15:01:36 and, that was my second point, contrib should be split before moved 15:02:06 hm how do we handle qa-test-tools? 15:02:14 they are also there... 15:02:19 hea etc. 15:02:19 (I am not voluntering) 15:02:31 what's the master repo? 15:02:33 cait: https://gitlab.com/koha-community 15:02:45 i know, was looking at kc 15:02:59 so we actually have to repos already for qa-test-tools 15:03:02 and hea 15:03:29 I think we should consider contrib abandnware, and use whatever we need, on gitlab 15:03:43 at the moment there's really not a clear aproach on how to contribute to those.. 15:03:52 that's where this question came from really 15:03:55 i feel like maybe make gitlab the main one... and mirror on kc? 15:04:17 people who contributes know how to contribute, and nobody else will contribute to contrib. So not a big deal 15:04:18 at some point i was submitting stuff to Joubu directly for sandboxes improvments I think 15:04:21 maybe more a question of how to handle access too? who can push etc 15:04:35 fair enough 15:04:54 * ashimema uses contrib solely for sandboxes 15:05:09 actions? 15:05:17 and working on them today highlighted at least documentation issues if not others (still digging) 15:05:31 i thnk we need someone to work out a votable suggestion maybe? 15:05:40 I think we should vote this on a general meeting 15:05:49 how we handle satellite repos 15:05:49 yeah.. agreed 15:05:50 vote what? 15:05:50 agreed 15:06:00 defer to a general meeting 15:06:10 I moved all of them to gitlab/k-c, it has been voted already 15:06:20 or at least done, 90% of them 15:06:35 ok, see later 15:06:40 #topic Set time of next meeting 15:06:43 * ashimema would like to see bywaters docker based sandboxes invested in a bit more.. they feel like the next logical step 15:06:54 ashimema: indeed! 15:07:24 Next meeting: 22 August 2018, 20 UTC? 15:07:25 okies 15:08:00 works for me 15:08:04 #info Next meeting: 22 August 2018, 20 UTC 15:08:04 works for me 15:08:06 #endmeeting