15:03:24 #startmeeting Development IRC Meeting 11 January 2023 15:03:24 Meeting started Wed Jan 11 15:03:24 2023 UTC. The chair is tuxayo. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:03:24 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:03:24 The meeting name has been set to 'development_irc_meeting_11_january_2023' 15:03:48 #link https://wiki.koha-community.org/wiki/Development_IRC_Meeting_11_January_2023 Today's agenda 15:03:56 #topic Introductions 15:05:04 #info Martin Renvoize, PTFS Europe, UK 15:05:08 #info Paul Derscheid, LMSCloud GmbH, Germany 15:05:37 #info Lucas Gass, ByWater Solutions 15:05:44 #info Emily Lamancusa, MCPL, USA 15:07:06 qa_team? 15:07:07 qa_team is cait, marcelr, khall, kidclamp, lukeg, aleisha, fridolin, ashimema, tuxayo, nugged, petrova, Joubu and dcook 15:07:32 rmaints? 15:07:32 rmaints is, like, liliputech, wainui, lukeg and ashimema 15:08:34 #info Katrin Fischer, BSZ, Germany 15:08:34 #info Nick Clemens, ByWater Solutions 15:08:42 Our security release was accounced today! 15:08:55 huzzah! 15:09:19 #info Security releases were accounced today! 15:09:30 🙂 15:09:44 Meeting bot is here, so hopefully the meeting will log ok 15:09:44 tests need some looking into 15:09:49 #info Jonathan Druart 15:09:54 #topic Announcements 15:10:27 I have not investigated but regarding Jenkins failures I would not upgrade 21.05 15:10:43 There is a compil error on C4::Auth 15:10:43 Thanks for the warning! 15:10:54 Anything else to announce that doesn't fit better the other topics? 15:11:33 Logging doesn't work unfortunately: https://irc.koha-community.org/koha/2023-01-11#i_2469114 15:12:50 tuxayo: meeting bot is separate 15:12:57 #topic Updates from the Release Maintainers 15:13:02 rmaints? 15:13:02 well, rmaints is liliputech, wainui, lukeg and ashimema 15:13:10 cait1: ah cool! 15:13:43 looking good there: https://meetings.koha-community.org/2023/ 15:14:30 #info Kyle M Hall, ByWater Solutions 15:14:35 Perfect :D 15:16:00 team++ for the security release 15:16:42 I am not about the ++ really, it was a nightmare to coordinate, and now the builds are failing 15:16:54 #info update your installation, see https://koha-community.org/ for infos about the security release. 15:17:26 what can we do to improve the process? 15:17:35 better documentation? hand-over for RMaints? 15:18:02 OK.. rmaint report wise 15:18:05 more communication, read the comments on bugzilla, don't ignore jenkins failures! 15:18:21 Communication has been an issue with the release team 15:18:21 And... dont' publish security bugs when jenkins is failing 15:18:32 #info Security releases have gone out, but it was a challenge to co-ordinate release over xmas period. 15:18:53 was the problem that jenkins didn't run on the changes because they were on the security branches? 15:19:05 so no access? 15:19:17 then run the prove commands locally? 15:19:19 We need to make sure rmaints know how to run the full test suit locally for security 15:19:25 you can't rely on Jenkins 15:19:45 I ran ours locally, but there appears to be failures on Jenkins.. I only ran a subset of tests it appears. 15:20:17 In general, I think rmaints should have a retrospective and talk about how we can do this better next time. 15:20:37 Yes 15:21:12 yeah, but maybe we could also make a note on the wiki: extra care torun all tetst locally 15:21:15 there are some key skills that need a bit of work.. we can get there together.. I intend to liase a bit with people and try to co-ordinate some shared learning. 15:21:17 for security 15:21:19 or so 15:21:22 yup 15:21:30 > don't ignore jenkins failures! 15:21:30 It's not part of the process to run the tests locally. So it can be forgotten. Anyone to complete the RMaint doc? 15:21:58 I can add a general note, but if there is a trick to run "all" someone else might want to add that 15:22:34 * ashimema has really struggled to keep on top of everything lately.. juggling new employees, support demands and then trying to help co-ordinate release and community actions on top. 15:22:51 new employees are awesome btw... all my queues are moving again 🙂 15:22:57 > if there is a trick to run "all" 15:22:57 run from the kohadev user. On root some tests will fail. 15:23:09 https://gitlab.com/koha-community/koha-misc4dev/-/issues/58 - this will help. Before it's done you need to replicate ktd/run.sh 15:23:12 but there's always a learning curve for a new team onboarding.. be that internally or in community. 15:23:48 yup.. we need to get that bug sorted Joubu.. that'll help a lot 15:24:15 it's almost at the top of my list.. 15:25:46 Joubu++ 15:26:07 #action cait add to RMaint documentation to run the test suite locally for security release 15:26:53 we need more spread of knowledge somehow too... right now I feel like Tomas, Joubu, Mason or I are all a little bit overly relied upon.. we need to try and spread our pooled knowledge a bit further so when one of us is out of action we're not so stuck as a community. 15:26:55 I just did: Important: As Jenkins won't run on the changes in the security branches, it's important to run the full test suite locally to avoid any unnoticed failures! 15:27:03 hope I put it in the right spot 15:27:10 last time I looked at run.sh, the core of it was prove -r t/ xt/ -j $(nproc) --rules='seq=t/db_dependent/**.t' 15:27:24 right now.. if mjt is out we're stuck for packaging for example 15:27:57 but there's a bit of a lack of volunteers to take on learning some of that knowledge.. 15:28:06 we need some people to step up and help with CI 15:28:09 Packaging 15:28:22 dev tools in general 15:28:22 I'd be up to write docs. 15:29:03 cait1: I am not blaming anyone, but jenkins is failing for stable branches for a while, it's not because of the security branches 15:29:35 just that people don't mind about jenkins failures 15:29:45 I understand, but that might go somewhere else in the tasks/docs 15:29:52 and now that the failures are not even on this channel it's even worse it seems 15:30:03 I also feel like we have some people who know stuff, but we don't have a lot of good documentation to point people to 15:30:09 it's something we shoudl focus a bit more on 15:30:29 the wiki often has contraditcting informration spread on many pages for some stuff, need to reduce that and find the 'truth' in there 15:30:44 I was pro the bot move.. but I think we did move too much.. I think some noise was good on here.. 15:30:47 I plan to maybe tackle some topics and compile some list of those pages - but I haven't fully formed a plan yet 15:30:55 like Jenkins failures I feel would be good to have 15:31:10 but will for sure need some help 15:31:35 +1 15:31:56 > like Jenkins failures I feel would be good to have 15:31:56 That still require all the rmaints get the chat here for failures. 15:32:31 Or maybe the point is so someone else ping them about the builds still failing? 15:33:06 no, rmaints should be responsible watching jenkins build after they push to their branch 15:33:41 * ashimema uses the dashboard 15:33:55 I check for failures there regularly for our branch 15:34:23 > rmaints should be responsible watching jenkins build after they push to their branch 15:34:23 +1, having the current process followed would be the best 15:36:13 there are indeed some builds that have been failing for a long time :o 15:37:09 re: ashimema's point on knowledge spread, I'm still just starting to learn how to develop Koha in general and asking newbie questions, but ultimately I'd love to learn from the pooled knowledge and help with documentation 15:37:18 hm yeah, maybe we need to be more strict and revert in those cases if possible 15:37:52 emlam: let me know if I can help you with documentation - it's good to know where/waht we are missing specifically or most urgently 15:38:04 i might also be able to dig something up from knowing where to look 15:38:16 (which is not how it should work) 15:38:30 cait1++ 15:38:55 likewise.. if you can't find cait I might be able to point in the right direction too 15:39:18 ashimema++ 15:40:18 I assume you're talking about dev documentation? 15:40:21 So ok to plan so discussion on completing the process and emphasizing the need to not less failing CI? 15:40:33 * caroline not trying to recruit or anything... 15:40:34 not less... no failing :) 15:40:46 rmaints? 15:40:46 rumour has it rmaints is liliputech, wainui, lukeg and ashimema 15:40:46 ↑↑↑ 15:40:52 (completing the process: the rmaint doc) 15:41:06 moving on? 15:41:30 actions for the failures? :D 15:41:47 right now people should not upgrade to 21.05, and sec releases have been announced 15:41:54 caroline: yes 15:42:00 #info Thomas Dukleth, Agogme, New York City 15:42:01 to latest 21.05 15:44:24 > actions for the failures? 15:44:24 > right now people should not upgrade to 21.05 15:44:25 I can reach wainui, any lead about how fix 21.05 while keeping the security fixes? 15:46:42 #action tuxayo reach wainui about the broken 21.05 release and investigate the failures. 15:46:47 ask the original author 15:47:03 don't ask somebody else to rebase 15:47:12 that would be my advise 15:49:10 > ask the original author 15:49:10 > don't ask somebody else to rebase 15:49:11 ok then I'll also reapply the 21.11 patch on a previous 21.05 to see if the backport was tricky. 15:49:14 thanks for the lead 15:49:34 lead/hint (I don't the right word) 15:49:40 #topic Updates from the QA team 15:49:47 qa_team? 15:49:47 it has been said that qa_team is cait, marcelr, khall, kidclamp, lukeg, aleisha, fridolin, ashimema, tuxayo, nugged, petrova, Joubu and dcook 15:50:21 #info Queues are quite full right now, I am pushing for the bad bugs and the old bugs... need more hands - please QA! 15:50:23 tcohen around? For next topic to be "Update from the Release manager (23.05)" 15:50:25 poking rabbit/background stuff thanks to dcook and Joubu for lots of work 15:50:45 yeah, that's totally out of my league and important 15:51:02 and easily bad bugs too 15:51:21 #info Work ongoing for rabbitMQ and bakground job improvements and fixes 15:51:34 jus tnoticed that it's quiet now, in theory we have a big QA team ;) 15:53:31 bug 32481 15:53:31 04Bug https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32481 critical, P5 - low, ---, dcook, Signed Off , Rabbit times out when too many jobs are queued and the response takes too long 15:53:40 bug 32393 15:53:40 04Bug https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32393 normal, P5 - low, ---, jonathan.druart+koha, Needs Signoff , background job worker explodes if JSON is incorrect 15:53:43 * cait1 hands over to Joubu :) 15:53:51 bug 32594 15:53:51 04Bug https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32594 enhancement, P5 - low, ---, nick, In Discussion , Add a dedicated ES indexing background worker 15:53:53 bug 32594 15:54:12 Should be high priority for everbody 15:54:57 bug 32395 15:54:57 04Bug https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32395 enhancement, P5 - low, ---, koha-bugs, In Discussion , Why do we send jobs params to the broker 15:55:24 and others, but those ones should go first. 15:56:57 thx 15:57:00 ready to move on? 15:57:03 thanks! 15:57:09 #topic Status of roadmap projects 15:57:45 Yes, I needed to find an email 15:57:47 #info roadmap: https://annuel.framapad.org/p/Roadmap_for_Koha_v23.05 15:58:23 tcohen said in an email in koha-devel: 15:58:24 «Now, it is time for us to build a new one ofr this cycle. Please fill with the things you plan to work on this cycle, and put your name so we all know and can help each other.» 15:59:33 #topic Actions from last meeting 15:59:55 Still have extra work now but next month, should test a new incremental version of MediaWiki Canasta container. 15:59:57 «tuxayo: Open tickets on KTD for the basic stuff needed to easily work on the non-root user.» 15:59:57 done 16:00:25 «tuxayo: Call for a volunteer on mailing lists to open the bidding for KohaCon24 and confirm the task list for that. » 16:00:25 Done, but I should also sent to the general list 16:01:06 #action tcohen To schedule/call a "CSRF day" to work on related patches together (previously deferred until after the 22.11 release). 16:01:28 #action tcohen To draft proposal for master→main change and draft of action list (previously deferred until after the 22.11 release). 16:01:46 «oleonard ashimema: Draft in the developer handbook how to form our modals. With template of a simple case and a case with form inside. Update: focus is currently on staff interface redesign changes, see the draft style guide: https://annuel.framapad.org/p/koha-styleguide-draft (previously deferred until after the 22.11 release).» 16:01:48 Anything on this? 16:03:37 #action oleonard ashimema Draft in the developer handbook how to form our modals. With template of a simple case and a case with form inside. Update: focus is currently on staff interface redesign changes, see the draft style guide: https://annuel.framapad.org/p/koha-styleguide-draft (previously deferred until after the 22.11 release). 16:03:49 #topic General development discussion (trends, ideas, ...) 16:04:26 Use of Gitea: We use Gitea for git.koha-community.org. Gitea was "forked" and is now Forgejo[1] - see the links update and related links[2] (sounds eerily familiar 8-(.. ). The community needs to look at what to do next, such as whether to continue to use Gitea, update to Forgejo, do something else.... (David Nind) 16:04:29 [1] https://forgejo.org/ 16:04:29 [2] https://forgejo.org/2022-12-26-monthly-update 16:04:43 oop.. sorry.. 16:04:44 back now 16:06:30 the ux guidelines are still moving forward slowly.. I need to grab oleonard and cait to discuss next steps 16:06:38 it's something I'd like to get formalised at hackgest 16:07:07 we can try that - might also be good to get feedback on the new style sin general there 16:07:22 tuxayo: forgejo is the 'good' one? 16:07:29 On that topic: Wouldn't it be useful to use somthing like penpot to visually formulate those guidelines in addition to written documentation? 16:07:58 cait1: that's what it looks like ^^ 16:08:50 nice idea paulderscheid 16:09:00 open-source + self-hosted 16:09:05 "After Gitea Ltd confirmed the takeover of the Gitea project on 30 October 2022, a group of people proposed that Codeberg e.V. should become the custodian of a fork of Gitea. The proposal was accepted 16 November 2022" - e.V. is a German thing actually 16:09:14 be lovely to host a community one 😜 16:09:35 what is a penpot? :) 16:09:56 https://github.com/penpot/penpot 16:10:22 Maybe we can host some stuff with LMSCloud's resources. 16:10:32 https://penpot.app 16:11:44 Added penpot mention in the styleguide draft 16:12:42 Hopefully there is a public instance hanging somewhere to try how it fits us without needing to host immediately. 16:13:29 There aren't really any alternatives, though. This is the only one I can think of, that really suits koha. 16:14:12 I mean, Figma is now part of Adobe. 16:16:14 Thanks for the hint paulderscheid and we will keep in mind that LMSCloud can host stuff ^^ 16:17:35 Anything else relating the styleguide? 16:17:38 And the trouble Gitea is going through. 16:17:43 Short addition before changing topics: https://www.plasmic.app/ 16:17:43 I guess I can also forward it to koha-devel. 16:17:43 Just glanced at it, though. 16:19:22 paulderscheid: that would be another tool the graphically document the styleguide? 16:19:48 #action tuxayo forward to the Forgejo/Gitea news and question to koha-devel 16:19:49 Ah, you can't self-host easily it seems. 16:20:03 Penpot is the only serious option, I think. 16:21:47 s/forward to the/forward the 16:22:42 #topic Review of coding guidelines 16:22:59 «SQL8 of our coding guidelines state that SQL should not be in .pl, but should be in the module in C4 or Koha. This would mean that SQL in the Koha namespace is allowed and there is no mention of DBIC in the coding guidelines right now. (Deferred from previous meeting(s).)» 16:23:16 cait completed with: «The question is: Is it allowed to use SQL statements in .pm modules in the Koha namespace? » 16:23:34 «The question here is if you can use SQL or if you should use dbic (not dbi) to make database queries. » 16:24:27 I think ashimema recently said on a bug we don't allow SQL in .pm in Koha 16:24:35 but I didn't find it in the coding guidelines 16:24:38 I'd like to add it 16:24:46 maybe as an addition to SQL 8? 16:25:14 https://wiki.koha-community.org/wiki/Coding_Guidelines «Other than those 2 cases, all SQL must be in the C4/xxxx.pm or Koha/xxxx.pm (xxxx being the module your SQL refers to) » 16:25:33 or Perl 20 16:25:34 that don't go with «we don't allow SQL in .pm in Koha» ^^" 16:25:37 PERL20 16:25:45 ecaxtly 16:26:03 I could have swarn it was in the guidelines 16:26:08 dang.. my memory is bad then 16:26:11 it sohuld be 😜 16:26:14 I'd say it should be? 16:26:34 i think the problem was that I phrased my question strangely, noone got what I meant with that agenda item when i was not around :) 16:26:46 so we could rewrite SQL8 16:26:46 hehe 16:26:54 and add a PERL20.2 16:26:57 there should be no raw sql, nowhere, isn't it what should be in the guidelines? 16:26:58 IIUC What is usually done with ORMs is that if there is SQL it can only be in the equivalent of our schema file. in the "custom section" 16:27:18 Joubu: i'd like that, but maybe we need to explain about not addig any new SQL 16:27:26 because we stil have C4/reports 16:27:47 yes, no new of course 16:27:50 I'dl ike something clearly spelled out to be able to point to it in QA of couse :) 16:28:18 hm and atomicupdates? 16:28:24 i think they actually need to use SQL to work? 16:29:19 Can we get away with not even needing to have a handcrafted SQL query in the DBIC schema files? 16:29:20 > hm and atomicupdates? 16:29:20 Ah yes lol! 16:29:45 Just don't be strict and we are good 16:29:56 you can write SQL queries if it's easier, or the only way to do 16:30:03 sometimes ORMs have limitations 16:30:18 that is not the clear line I was hoping for 16:30:18 but most of the time we don't need them 16:30:24 har dto tell sometimes when it's really neededor not 16:30:39 have we hit any restrictions that we could not get around yet with the ORM? 16:30:39 when you can do without, do without 16:31:01 otherwise i'd say we require DBIC 16:31:05 (without SQL?) 16:31:09 yes I remember writing some in a sub query or something 16:31:10 and people need to argue when they want to use SQL 16:31:35 or QA show them how to do without 16:31:51 I need to run, sorry. See you tomorrow 16:32:08 > sometimes ORMs have limitations 16:32:09 > but most of the time we don't need them 16:32:09 That's what I understood also. They all provide something like the section for custom queries that we have in our schema files. But it should be rarely used. 16:32:14 Joubu: Thanks for that clarificaton++ 16:32:47 thanks for the help Joubu , see ya 16:32:51 so... how can we improve the coding guidelines? 16:33:04 right now we don't even mention dbic... 16:34:43 So in the end: no SQL by default except for database upgrades. But if it's not possible to do a query with the ORM or if it's much less readable or a performance bottleneck, then it should be done in a custom subroutine in the relevant DBIC schema class. 16:35:30 Does that makes sense? 16:35:37 Yes 16:35:48 "Wherever possible SQL::Abstract syntax should be used over raw SQL in the /Koha namespace" 16:35:52 with a link to 'https://metacpan.org/pod/SQL::Abstract::Classic#WHERE-CLAUSES' 16:36:13 hmm 16:36:26 it find it pretty rare these days that I need to resort to raw SQL 16:36:43 very occasionally a string ref within a abstract syntax 16:36:48 \"" 16:36:52 \\"" 16:37:29 the advantages of doing it as SQL::Abstract are that DBIC understands it and can optimise things behind the scenes for you.. 16:39:06 So SQL::Abstract is how to construct complex queries outside of the CRUD automatically provided by the ORM? 16:40:21 And without having to type raw SQL. 16:40:25 SQL::Abstract is the syntax for queries etc that DBIC is based on top of 16:40:28 it's the name for the dbic query structure 16:42:25 "Wherever possible SQL::Abstract syntax should be used over raw SQL in the /Koha namespace" 16:42:25 Should we also mandate that those queries are in custom subroutine in the DBIC modules? 16:43:36 To isolate the logic of getting data. And be sure to not duplicate it. 16:44:18 ashimema++ thx 16:46:20 hi all 16:46:52 hi mtj ^^ 16:47:48 Joubu: perhaps some good news? .. i forgot to build a new 21.05 pkg 16:48:09 He had to go already 16:48:10 yes good news actually XD 16:48:15 will you update the wiki? 16:48:58 #action tuxayo try to sum up ideas about guideline for SQL and sent to koha-devel for completion, rephrasing and terminology fixing. 16:49:04 🙂 16:49:10 ..so current 21.05 pkg is still 21.05.21, from nov 16:49:18 no worries 16:49:20 perhaps good 16:49:29 What is happening with channel logs? 16:49:35 i forgot 21.05 is now lTS 16:49:40 I feel we should get the tests passing consistently everywhere and re-roll a package 😜 16:50:05 > will you update the wiki? 16:50:05 The coding guidelines? 16:50:06 the coding guidelines is https://wiki.koha-community.org/wiki/Coding_Guidelines 16:50:41 > i forgot 21.05 is now lTS 16:50:41 mtj: It's not. It's just that we have enough volunteer that we can also maintain it. 16:50:49 It's a bonus :) 16:51:33 Who maintains IRC channel logging? 16:51:56 thd: the bot is getting kicked off - rangi was looking into it 16:52:31 thd: rangi and they where informed about the issue. 16:52:33 yes, what cait1 said 16:53:00 Still the meetings are getting logged via another boot :) 16:53:14 i have a backup of the channel logs, so we can rebuild them 16:53:26 mtj++ 16:53:59 mtj: does [alohalog](https://matrix.to/#/@_oftc_alohalog:matrix.org) have them published somewhere? 16:54:25 #topic ACK pre vs post processing for background jobs 16:55:02 meeting is getting a little long'ish 16:55:05 Unless there are a few people here very inspired about that, let's defer it. 16:55:07 should we try to wrap up? 16:55:20 actually i need to cook dinner :) 16:55:36 here was the content if someone wants to add someone that I relay to the next meeting: 16:55:40 «Bug 32573 - background_jobs_worker.pl should ACK a message before it forks and runs the job suggests we should ACK of STOMP messages before we begin processing which would avoid the issue 30 minute time issue among other problems. It appears that the STOMP documentation and the RabbitMQ documentation do not agree. We should have a vote if needed» 16:55:40 04Bug https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32573 normal, P5 - low, ---, kyle, In Discussion , background_jobs_worker.pl should ACK a message before it forks and runs the job 16:55:40 aah '22.11 will be the first official LTS Release. (Decision: Development IRC meeting 3 August 2022)' 16:56:17 #topic Set time of next meeting 16:56:49 for this timeslot: same hour the 8th february? 16:57:10 👍️ 16:57:21 mtj: yes, we hope to keep that around for a long while :) 16:57:21 #info Next meeting: 25 January 2022, 23 UTC 16:57:27 backup logbot is https://ilbot3.kohaaloha.com/koha/ 16:57:29 that's for the other timeslot 16:57:44 8th shold be ok 16:58:03 mtj: thanks a lot, that can help when the other is down :) 16:58:37 mtj: can I add the link in the IRC_Regulars with alohalog` ? 16:59:10 ######info Next next meeting: 8 February 2022, 15 UTC 16:59:15 tuxayo: yes ok 16:59:20 #endmeeting