Talk:SAC Meeting 2018-05-27

   06:55:15	robe2:	https://wiki.osgeo.org/wiki/SAC_Meeting_2018-05-27
   06:55:16	sigabrt:	Title: SAC Meeting 2018-05-27 - OSGeo (at wiki.osgeo.org)
   06:59:52	MartinSpott:	Voilaaaa ! ;-)
   06:59:59	astrodog:	Ha.
   07:00:06	MartinSpott:	Oh my, it's Sunday morning ....
   07:00:45	robe2:	Everyone ready for meeting?
   07:00:52	robe2:	I'm sure strk is snoozing
   07:00:54	astrodog:	I'm around at the moment on a Marketing Committee set of questions I've got. Cool to see the new hardware progressing, though!
   07:01:32	robe2:	astrodog should I add to list?
   07:01:47	robe2:	or is it on already. At a glance don't see marketing in there
   07:01:54	MartinSpott:	no
   07:02:04	astrodog:	robe2: Nah, just an informal thing. Making sure some things are realistic before I propose them.
   07:02:23	robe2:	astrodog great :)
   07:02:41	MartinSpott:	indeed
   07:03:10	astrodog:	Better to have that discussion ahead of time than get everyone all hyped up, then realize there's no way we can support it. :P
   07:03:32	robe2:	First on list hardware - well it's shipped, but not sure if OSUOSL has got it yet
   07:04:14	robe2:	TemptorSent did you and wildintellect ever get around to outlining the plans for new server?
   07:04:35	MartinSpott:	moreover I'd expect a few days delay for racking, connecting some remote console
   07:05:13	robe2:	Okay seems TemptorSent is asleep to move on to next topic
   07:05:39	robe2:	osgeo6 I lost track of what's going on with that - we had disk issues and that coin minining thing
   07:05:58	robe2:	hard disk
   07:06:10	MartinSpott:	disk issues on osgeo6 itself ?
   07:06:13	robe2:	MartinSpott what were your thoughts on that
   07:06:27	robe2:	sorry not disk more how we run out of disk and doing net whatever on it
   07:06:35	astrodog:	I do have a thought on osgeo6... did we ever determine if the compromised account was also capable of delivering or altering content?
   07:07:14	robe2:	astrodog no clue -- it was calling out to germany some where was all I could surpmise
   07:07:15	MartinSpott:	you mean the geotools account ?
   07:07:21	astrodog:	Yeah.
   07:07:23	robe2:	yes geotools
   07:08:42	MartinSpott:	as far as can tell: no
   07:08:42	astrodog:	Assuming the account has the ability to deliver content to users... we should probably write something up. Even if there's no indication that it actually did anything, someone just stumbling onto that issue browsing the lists and not seeing anything anywhere else is bad times.
   07:09:15	robe2:	who owns the geotools account?
   07:09:54	MartinSpott:	no idea
   07:10:21	astrodog:	Even if it's just a page or so of "This account was compromised, we have not seen any indication that it delivered malicious binaries or content to users, or extracted any user information. We've fixed it by doing 'X'." etc, etc.
   07:10:22	robe2:	did we disable it already guess someone will complain when we do :)
   07:10:23	MartinSpott:	the account says: geotools Build User Boundlessgeo
   07:11:06	astrodog:	Have we identified how it was compromised, or is the guess a bad PHP script?
   07:11:18	robe2:	so jive[m] might know something about that account
   07:12:08	robe2:	astrodog no not sure how it was compromised. I don't even guess a bad PHP script
   07:12:12	MartinSpott:	looks like an account to run some builds, the command history mentions Confluence
   07:12:39	astrodog:	What / who do we need to get what happened there run down?
   07:14:04	robe2:	astrodog well it was Markus Neteler that discovered it
   07:14:09	MartinSpott:	From naive view I'd say this account is being used to update some geotools public web page from SVN using an automated process
   07:15:20	robe2:	I'll send Jody a note he's the only one I know on that team
   07:15:32	MartinSpott:	But I have to admit I didn't see the mysterious processes which have been observed by others
   07:15:53	MartinSpott:	simply because I wasn't around in time
   07:15:59	robe2:	MartinSpott it was a process running in a folder " "
   07:16:04	astrodog:	Since we do actually deliver a fair number of binaries and the like to people... we really just need to figure out what we're doing with it and say that publicly.
   07:16:06	MartinSpott:	ah
   07:16:10	robe2:	so essentially hidden folder in tmp folder
   07:16:19	robe2:	and it was called d or something like that
   07:16:41	robe2:	researching it was a bitcoin harvester executable I think released in march or so
   07:16:46	robe2:	I forget the exact date
   07:17:11	MartinSpott:	ok
   07:17:56	robe2:	So what would be the best course of action - disable the account entirely and have geotools setup a new one? Do nothing? just reset the password?
   07:18:04	astrodog:	Martin: Do you happen to recall what OS and kernel version are running on that host?
   07:18:06	MartinSpott:	in this case I'd agree with setting a deadline for password renewal
   07:18:30	robe2:	I guess the other concern is if that is the only one compromised on that server isnce it was discovered by accident
   07:18:32	MartinSpott:	astrodog: It's Debian8 and kernel 4.7
   07:19:27	robe2:	MartinSpott is there any way to enforce password renewal? I'll ask jody if he can get someone to reset the geotools one
   07:19:34	MartinSpott:	robe2: geotools is not in the sudoers list
   07:19:49	astrodog:	Martin: Any indication that it's spread beyond that account? 4.7 is old enough that I'd expect some random local escalation exploits to be floating around.
   07:19:53	robe2:	so we don't care so much about it
   07:20:04	MartinSpott:	well, we can simply set a new one and the the user ask for it :-)
   07:20:13	MartinSpott:	and *let* the user
   07:20:40	astrodog:	Before we bring the account back online, we kinda need to figure out how it got there in the first place. :\
   07:21:18	MartinSpott:	Most communication in this account is authorized by public keys, I'd say
   07:21:21	robe2:	MartinSpott not following *let* the user?
   07:21:38	robe2:	ah okay so resetting shouldn't be an issue
   07:21:42	astrodog:	robe: I think Martin is saying, break the account, they'll complain.
   07:21:47	robe2:	resetting the password I mean
   07:22:07	robe2:	MartinSpott okay can you break it?
   07:22:18	robe2:	I can give Jody a heads up at least :)
   07:22:33	MartinSpott:	astrodog: correct
   07:22:43	astrodog:	If the account is primarily driven by SSH keys, that may be how it got in, too. Could be someone with access to it had an issue.
   07:23:44	MartinSpott:	Let's reset the password plus remove the SSH keys and announce their removal in advance to some list which is linked to geotools
   07:24:21	robe2:	Sounds good
   07:24:21	MartinSpott:	I'm sure they'll understand if we tell them why
   07:24:27	astrodog:	Putting on my... TrustedBSD contributor hat for a moment, my inclination would be to disable the account, and work with the geotools folks to figure out what happened to it. Just resetting its keys and passwords doesn't buy us much if we don't know what happened.
   07:25:11	MartinSpott:	that's fine with me as well
   07:25:48	astrodog:	I'd probably try and coordinate that off-list, we can do some kind of announcement when we've got meaningful details to share. As it stands, it seems like a very minor intrusion and someone stumbled on to the machine.
   07:25:50	MartinSpott:	who knows where to find these geotools folks ?
   07:25:57	astrodog:	They're on IRC, I think?
   07:26:14	astrodog:	Jody is, anyway.
   07:26:38	robe2:	I'm about to send Jody an email. the list looks like sourceforge
   07:26:49	MartinSpott:	ok, let's check mark this item
   07:26:56	astrodog:	I'll ping him on Skype, too.
   07:27:17	astrodog:	Martin: Do you want me to coordinate running it down with them?
   07:27:26	MartinSpott:	please ;-)
   07:27:40	robe2:	so no do and ask questions later :)
   07:27:49	robe2:	you guys are wusses
   07:28:21	astrodog:	robe: Nuke osgeo6 from orbit, it's the only way to be sure. Better?
   07:28:22	astrodog:	:P
   07:28:29	robe2:	yes
   07:28:32	robe2:	that's more like it
   07:28:50	robe2:	I'm sending Jody a note saying we are shutting it down and talk to his people
   07:28:59	robe2:	cc'ing Martin and Alex
   07:29:03	astrodog:	Martin: Do we have any sort of binary integrety checking on osgeo6?
   07:29:03	MartinSpott:	great
   07:29:30	MartinSpott:	as far as I can tell, we don't
   07:29:48	MartinSpott:	you men tripwire or the like ?
   07:29:56	astrodog:	Yeah.
   07:30:05	MartinSpott:	pretty sure we don't
   07:30:22	astrodog:	You mind doing a basic runthrough? Statically linked sha binary, and just compare $PATH and lib to whatever is in the distribution.
   07:31:07	MartinSpott:	yes, I'll fetch one to check the current system binary set for known suspects
   07:32:05	astrodog:	That'd be great. There are a few 1-2 line scripts using find -x that can crank through it pretty quickly.
   07:32:43	robe2:	MartinSpott sent note to Jody and cc'd you
   07:34:03	MartinSpott:	astrodog: Noted in the ticket so I don't forget ....
   07:35:19	MartinSpott:	robe2: Next item ?
   07:36:05	robe2:	Next item is funtoo but we were waiting for board to test the nextcloud thing
   07:36:08	robe2:	we can skip that
   07:36:24	robe2:	oh I know the question I had
   07:36:35	MartinSpott:	next one is download's disk extension vi NFS
   07:36:36	robe2:	MartinSpott you know if there is an ldap group for board members?
   07:36:47	robe2:	I couldn't find one
   07:36:49	MartinSpott:	ldap for board, no idea - wait a second
   07:37:09	robe2:	that was on my list to just limit to board but couldn't find such a group
   07:38:11	MartinSpott:	no, no group whose name relates to board
   07:38:25	MartinSpott:	or resembles with
   07:38:39	robe2:	seems like we should have one. I'll put in a ticket to add one
   07:38:43	astrodog:	Propose we make a board LDAP group!
   07:38:44	astrodog:	:P
   07:38:46	MartinSpott:	fine
   07:41:29	robe2:	https://trac.osgeo.org/osgeo/ticket/2170
   07:41:30	sigabrt:	Title: #2170 (Create LDAP group for board) – OSGeo (at trac.osgeo.org)
   07:42:50	robe2:	grrh I've been editing the wrong sac meeting
   07:43:04	astrodog:	These kinds of meetings are quick when you only have 3 attendees and one of them only shows up once every year or two. :P
   07:43:47	MartinSpott:	I'll set it to Milestone Sysadmin Contract for better visibility
   07:44:23	robe2:	next topic nfs mount
   07:44:45	robe2:	how much disk space is qgis and (what is the other one, I forget) taking up on nfs mount?
   07:44:59	robe2:	oh osgeo4w
   07:45:11	astrodog:	robe2: Sorry if I'm rehashing something that's been discussed before, but is there a particular reason to use NFS for this versus an iSCSI target?
   07:45:35	MartinSpott:	both approx 30 GByte
   07:45:37	MartinSpott:	each
   07:45:38	astrodog:	(Or, is that just how it's set up, and there's no particular need / interest in poking at it?)
   07:45:47	robe2:	astrodog no clue, I think it was one of those in the moment decisions "This will work" so it was done
   07:45:59	astrodog:	Gotcha.
   07:46:06	MartinSpott:	yup
   07:46:20	MartinSpott:	what make me so upset is:
   07:46:23	robe2:	I'm not familiar with iSCSI target
   07:46:46	astrodog:	There are some... performance and management gains to be had in changing that, but I don't know where that would fall on the list in terms of SAC priorities.
   07:46:54	astrodog:	robe: Block device over the network, basically.
   07:46:58	robe2:	MartingSpott sorry for interrupting
   07:47:10	MartinSpott:	a) people are excercising root permissions while having no idea of security concerns
   07:47:22	MartinSpott:	b) simply because it's convenient
   07:47:53	MartinSpott:	c) even though it adds maintenance burdeb
   07:48:00	MartinSpott:	d) period
   07:48:13	robe2:	people will be people - take the easiest road to immediate satisfaction
   07:48:24	robe2:	gratification I should say
   07:48:35	MartinSpott:	NFS isn't bad in general, but apparently there are too many users having sudo permission
   07:48:43	astrodog:	One of the advantages of using the iSCSI target thing is that it blocks the... permissions flow through.
   07:48:51	MartinSpott:	yup
   07:49:11	astrodog:	You can have the filesystem be r/o on the downloads server, and let the actual file management happen somewhere else.
   07:49:14	robe2:	so is it hard to set up or even worth it if some of that will be offloaded to osgeo7
   07:49:30	astrodog:	It's pretty easy, but I'd probably defer it to the new server setup.
   07:49:33	robe2:	astrodog so sounds like a good long term solution
   07:50:50	robe2:	I aslo saw ftp running on osgeo6 which surprised me
   07:50:57	robe2:	looked like plain old ftp
   07:51:15	astrodog:	I think some things download over that, osgeo4w comes to mind.
   07:51:27	robe2:	over ftp instead of http?
   07:51:32	astrodog:	Yeah.
   07:51:47	astrodog:	You can definitely feed the installer FTP mirrors, anyway.
   07:51:59	robe2:	okay that explains it
   07:52:23	astrodog:	Might be worth seeing if anyone uses it, though.
   07:52:23	robe2:	and I think someone said we have webdav too presumably for maven
   07:52:58	robe2:	it'd be nice if download was just downloads
   07:53:37	robe2:	next topic wiki / ldap
   07:53:39	astrodog:	We could do that as part of the iSCSI change. We'd be doing some cleanup on file layouts anyway.
   07:53:51	robe2:	MartinSpott did you ever create pgbackup sorry if I missed it
   07:54:09	MartinSpott:	err, pgbackup ?
   07:54:14	astrodog:	Oh god, wiki and LDAP is still a thing? I know I dropped the ball on that... but jeez.
   07:54:41	robe2:	astrodog a thing you want it to not be a thing?
   07:55:23	astrodog:	robe2: I want to light the entire pile on fire, push it on a boat into the sea, and we can honor it once a year by drinking heavily.
   07:55:26	MartinSpott:	I've created dumps to be sent to TemptorSent
   07:55:30	robe2:	MartinSpott pg dump of wiki database
   07:55:58	robe2:	MartinSpott can i have access to them too?
   07:55:59	MartinSpott:	MySQL plus filesystem backup of the Wiki dir
   07:55:59	astrodog:	Is TemptorSent coordinating that one now?
   07:56:38	MartinSpott:	astrodog: He suggested to set up a copy of the Wiki for test with LDAP authentication and user migratio
   07:56:43	robe2:	MartinSpott is it running on MySQL? I thought it was on postgres.
   07:57:03	MartinSpott:	MySQL, unfortunately, no Postgres
   07:57:09	robe2:	:(
   07:57:31	robe2:	are we planning to run the new one on Postgres or we going to try a two phase migration?
   07:57:39	astrodog:	Martin: Ah, gotcha. I'll track down my notes and pass them along. As I recall, where things kinda fell apart was trying to figure out what the relationship of wiki and LDAP users might be.
   07:57:48	MartinSpott:	People more familiar with the topic than I am suggested to stick with MySQL because some extentions would probably not work on Postgresql
   07:58:17	robe2:	MartinSpott yah I was afraid of that
   07:58:30	MartinSpott:	sounds like some adhere to the MySQL dialect of SQL
   07:58:42	MartinSpott:	or the like
   07:58:59	astrodog:	Do we use any extensions like that? (Or, do we know?)
   07:59:32	robe2:	astrodog I have no clue. Doesn't seem like we use wiki for much aside from wiki
   07:59:42	robe2:	so I would think we are minimalist on extensions or should be
   07:59:55	MartinSpott:	we're using quite a few extensions and I don't have in-depth knowledge of the respective implications
   08:00:07	astrodog:	Maybe that'd be something else to do in the test... see if it breaks if we switch it,
   08:00:31	MartinSpott:	there's a list on the Wiki I took as a reference during the last upgrade
   08:00:32	robe2:	LDAP is already a big mouthful though
   08:00:59	robe2:	as much as I hate MySQL I'd stick with it (well MariaDB at anyrate)
   08:01:13	MartinSpott:	same here
   08:01:51	robe2:	MartinSpott still I'd like to have access to the mysql backups
   08:02:08	MartinSpott:	where sould I put it ?
   08:02:12	astrodog:	I suppose I'm agnostic on it, really. It seems strange that we'd be using something that would preclude a change, but exactly what database sits behind the wiki seems, ultimately, pretty immaterial.
   08:02:37	robe2:	I actually have access to the wiki server I think
   08:02:41	astrodog:	(Also, my opinion on it doesn't count for much. I'm not stuck maintaining it, you guys are.)
   08:02:51	robe2:	so if you put it somewhere there or even on osgeo6 that would be fine
   08:03:07	MartinSpott:	robe2: it already is
   08:03:10	robe2:	on osgeo6 I have sudo but wiki I think I just have regular access
   08:03:18	MartinSpott:	because disk space on the Wiki is getting tight ....
   08:03:36	MartinSpott:	robe2: wait a second, I'll move it into your home dir
   08:04:26	MartinSpott:	there's a wiki page on the wiki plugins we use for the wiki (hah, three times "wiki" in a single, short sentence)
   08:04:42	MartinSpott:	where is it ? ....
   08:05:02	robe2:	MartinSpott I don't have a home directory on wiki server
   08:05:18	MartinSpott:	on osgeo6
   08:05:25	robe2:	oh okay
   08:06:26	robe2:	weird I don't seem to have a home on osgeo6 either
   08:06:37	MartinSpott:	we'll make one
   08:07:14	robe2:	okay make one and put it in there
   08:08:33	MartinSpott:	next topic ?
   08:09:30	robe2:	MartinSpott I think we are done other topics involve people not here
   08:09:40	robe2:	astrodog wanted to raise your questions
   08:09:51	robe2:	marketing stuff I guess is the next topic :)
   08:10:13	astrodog:	My marketing related question is: How much of a burden would it be if Marketing wanted to work with the constituent projects to set up some coordinated demos?
   08:10:41	robe2:	astrodog burden for who?
   08:10:43	astrodog:	A stripped down version of something like maps.esri.com, or some of their examples.
   08:10:45	astrodog:	SAC.
   08:10:51	robe2:	sounds like more of a burden for marketing :)
   08:11:19	robe2:	astrodog depends how much active content and stuff needs installing
   08:11:23	astrodog:	You say that, until you try to run QGIS's WMS server in production, and realize you're tracking Qt dependencies on a webserver. :P
   08:11:25	robe2:	setting up a domain would be easy
   08:12:13	robe2:	astrodog too bad TemptorSent isn't here
   08:12:30	robe2:	I was thinking our funtoo server might be a good fit for that
   08:12:45	robe2:	we could setup containers for each of those
   08:12:54	robe2:	lxc containers
   08:13:10	astrodog:	My guess would be a domain plus services. We can keep the datasets small, so the actual memory, data and CPU footprint stays managable... but I think we're really missing something by not showing how the OSGeo projects work together.
   08:13:46	robe2:	astrodog I agree I like the idea
   08:14:11	MartinSpott:	I'd say it depends on how much self contained the setup would be
   08:14:58	astrodog:	I think we can keep it very self-contained... it'd mostly be showing how the various pieces combined with OGC standards let you do spatial stuff.
   08:15:11	MartinSpott:	if someone is around to put everything into a container, then I'd say go for it
   08:15:13	astrodog:	Ideally, we'd also work out some... combined installation documentation at the same time.
   08:15:21	astrodog:	I don't want to suggest it to marketing if it's going to drown SAC, though. How would you guys feel about trying to do a quick proof of concept when the new server is up?
   08:15:25	MartinSpott:	but probably you're reaching out for such person right now
   08:15:54	astrodog:	Martin: Getting it up and running isn't too bad, but SAC would probably be stuck with keeping it up to date.
   08:16:05	astrodog:	(In terms of software versions)
   08:16:30	robe2:	astrodog so this would be an all encompassing thing showing all osgeo goodies
   08:16:35	MartinSpott:	does it rely on software installed on the "host" system or does it bring its own ?
   08:16:51	robe2:	so something like maps.osgeo.org with all the different pieces interoperating
   08:17:06	astrodog:	robe: That's what I have in mind, yeah.
   08:17:07	MartinSpott:	if it's self contained, than replacing an old bundle with a new one is cheap
   08:17:26	robe2:	I think we should run it in a container for sure
   08:17:31	astrodog:	Martin: I think we'd need to bring our own in the container. The dependency hell on the host would escalate quickly.
   08:17:36	MartinSpott:	maps.osgeo.org is dead
   08:17:46	MartinSpott:	I mean not existent
   08:18:17	astrodog:	Are you guys okay with me posting to marketing saying SAC is looking into the viability of the idea?
   08:18:22	robe2:	that's good so we can use the domain :)
   08:18:46	robe2:	astrodog I am but might want to wait for wildintellect's blessing on that one
   08:19:05	astrodog:	Marketing seems a little... downbeat these days, after pushing through the rebranding, and I think it'll be exciting for people.
   08:19:48	robe2:	well sounds exciting to me I know postgis we always wanted to do something like that
   08:20:14	robe2:	but it would be more of a try your queries kinda thing
   08:20:39	robe2:	okay meeting adjourned
   08:20:41	MartinSpott:	I think a long-term agenda we can call "containerize it"
   08:21:07	astrodog:	I have some cool PostGIS/QGIS/R use cases we could show. Pretty sure some clients would be fine with throwing their data in if we need that.
   08:21:11	robe2:	yah I think wildintellect and TemptorSent were supposed to flesh that out with Osgeo7
   08:21:20	MartinSpott:	robe2: what is "something like that" ? A site to compare diferent map renderers ?
   08:21:22	robe2:	I thought the plan was to run lxc containers on that
   08:22:09	robe2:	MartinSpott was talking about just containerize in general so we don't have to worry about dependencies clubbering each other
   08:22:56	robe2:	astrodog would we use something like Jupyter Notebooks
   08:23:30	robe2:	I see everyone talking about those things and saying how great they are especially for demoing and showcasing, but I haven't found the energy to learn yet another thing
   08:26:48	MartinSpott:	robe2: you're "robe" on osgeo6, correct ?
   08:27:00	robe2:	yes
   08:28:50	MartinSpott:	voila
   08:29:08	MartinSpott:	open topics to be discussed today ?
   08:29:57	robe2:	don't think any other topics to be discussed
   08:30:09	MartinSpott:	close the meeting ?
   08:30:22	robe2:	took us 1.5 hrs (well a little under) -- I'm pooped
   08:30:51	MartinSpott:	sleep well, thanks for tacing care of the meeting schedule - in particular as well as in general
   08:31:17	robe2:	MartinSpott you are welcome and thanks for all that system stuff you do that I don't understand :)
   08:31:20	MartinSpott:	my family woke up, time to play with the kids
   08:31:36	robe2:	by that I mean that we have someone as capable on our team as you
   08:31:46	robe2:	have fun with the kids ttyl
   08:32:23	robe2:	meeting closed
   08:32:27	robe2:	ttyl