Talk:SAC Meeting 2018-03-15

Transcript

   20:00:40	robe2:	Okay time for meeting
   20:00:52	robe2:	https://wiki.osgeo.org/wiki/SAC_Meeting_2018-03-15#Agenda
   20:00:53	sigabrt:	Title: SAC Meeting 2018-03-15 - OSGeo (at wiki.osgeo.org)
   20:01:00	robe2:	cvvergara want to moderate?
   20:01:01	jive[m]:	Time zones are difficult
   20:01:22	robe2:	We should just irradicate timezones
   20:01:40	robe2:	or at very least day light savings crap
   20:01:51	wildintellect:	EU is working on that
   20:02:42	robe2:	ping cvvergara
   20:02:42	markusN:	+1 to get rid of day light savings crap
   20:02:55	robe2:	:)
   20:03:28	markusN:	(I'm a bit on and off due to kids' peak hour here)
   20:03:53	robe2:	kids have a peak hour?
   20:04:12	robe2:	I thought their peak is defined as any time that is inconvenient for you
   20:04:31	wildintellect:	thats most of the time
   20:04:44	robe2:	okay first topic the Hardware
   20:05:14	robe2:	I admit I haven't been following all the back and forth on that so wildintellect can you summarize where we stand
   20:05:25	robe2:	I heard something about an optane card and got lost
   20:05:30	wildintellect:	Chris and I discussed some modifications to the hardware and I requested an updated quote
   20:05:52	robe2:	update quote from the vendor?
   20:05:55	TemptorSent:	RE: Hardware -- I did more digging into the the system SM is basing our config on and found that we are slightly less limited than had been thought.
   20:05:58	wildintellect:	the important part is I questioned the need for the expensive SSDs if they were just for filesystem caching
   20:06:23	wildintellect:	we don't have the new quote yet, I emailed them maybe 4-5 hours ago
   20:06:49	robe2:	okay so first topic done, we may end in 30 minutes this time :)
   20:07:01	wildintellect:	I did want to get a general feel for if people are comfortable the caching technologies discussed
   20:07:05	TemptorSent:	Short answer is that write caching is critical to responsiveness for DB type loads, read cache less critical.
   20:07:23	robe2:	so TemptorSent, palica, angry_vincent next topic the funtoo stuff
   20:07:35	wildintellect:	Can I get a raise of hands how many people have used ZFS before?
   20:07:39	TemptorSent:	palica isn't available for about an hour I believe.
   20:07:46		* TemptorSent raises hand.
   20:08:04	robe2:	TemptorSent you have the floor
   20:08:19		* robe2 feels like such a bureaucrat, that's what strk is supposed to be for
   20:08:34	TemptorSent:	*lol* Well, I was raising my hand RE use of ZFS :)
   20:08:48	robe2:	oh I missed that
   20:08:55		* robe2 raises hand for ZFS
   20:09:06	robe2:	oh wait no
   20:09:20	robe2:	sorry never used ZFS but would like to learn heard good things about it
   20:09:24	TemptorSent:	*lol*
   20:09:30	wildintellect:	good enough
   20:09:44	wildintellect:	if Martin is ok with it, it should go over fine
   20:09:53	wildintellect:	though we might have a debian v ubuntu debate
   20:09:58	TemptorSent:	Okay, let me give a brief explanation then regarding ZFS and how it handles caching I suppose.
   20:10:11	wildintellect:	links to the list should suffice
   20:10:14	robe2:	yah ZFS seems to be all the rage in the postgres community and I feel inadequate not understanding what they are talking about
   20:10:41	TemptorSent:	ZFS handles the storage stack from end to end, including the caching and ioscheduling for the drives.
   20:11:49	TemptorSent:	You can assign a fast write-cache known as a "SLOG", which will allow sync writes to finish as soon as they're written to the fast storage and be synced to the slower disks later.
   20:12:47	TemptorSent:	Because writes are not immediately stored to the redundant drives backing the pool, it's important that the write-cache is redundant as well.
   20:14:04	TemptorSent:	The read cache is known as the L2ARC, which augments the in-memory ARC (Adaptive Replacement Cache), keeping blocks evicted from RAM avaiable quickly without a disk seek.
   20:14:33	robe2:	ZFS works on debian/ubuntu now though right - that was my understanding from reading this article a while back - https://blog.2ndquadrant.com/pg-phriday-postgres-zfs/
   20:14:34	sigabrt:	Title: PG Phriday: Postgres on ZFS | Ways ZFS Can Make Postgres an Even Better Database (at blog.2ndquadrant.com)
   20:15:00	TemptorSent:	Yes, ubuntu actively supports zfs, debian treats it as a red-headed step-child.
   20:15:39	TemptorSent:	But it will work on either.
   20:15:43	robe2:	oh
   20:16:42	robe2:	my bad I keep thinking of ubuntu / debian as being much the same but forgot debian is filled with shall we says "It must be absolutely perfectly right" and Ubuntu is "Good enough, good enough"
   20:16:42	TemptorSent:	It's a philosophical issue, not a technical one, it just means that it's not in the main repo for debian whereas it is on ubuntu
   20:17:37	TemptorSent:	And if you wanted to get a support contract through Cannonical for zfs-on-ubuntu, you could :)
   20:17:48	robe2:	Is it in like the sid repo of debian or not even that
   20:18:06	TemptorSent:	It's in sid, just not in the main repo.
   20:18:11	wildintellect:	thats probably enough detail on that topic for now
   20:18:47	TemptorSent:	Short answer - ZFS on Ubuntu is actively supported by their parent company.
   20:19:52	TemptorSent:	LXD is also actively supported by them, so I would suggest based on the general comfort with debian based tools that ubuntu be considered for the new hw's primary environment.
   20:20:00	robe2:	but we can get from sid. Many repo is so old anyway I never bother with it
   20:20:01	jive[m]:	(reading adgenda I have nothing to contribute, I will note that we have put "osgeo website" on the docket for the code sprint next week; hope to set attendees with OSGeo id and website access over the course of the week)
   20:20:32	robe2:	jive[m] great -- is new server working okay for you?
   20:20:50	robe2:	I'm going to shut down teh cloudvps at end of this month since we are no longer using it
   20:21:28		* markusN back & reading
   20:21:39	TemptorSent:	Okay, for Funtoo, I'll give a brief explanation and hopefully palica can chime in later with more details from the staff side.
   20:21:57	robe2:	okay enough detail so we can ask Martin and others on list again about their comfort level with ZFS
   20:22:14	robe2:	as far as debian/ubuntu I'm pretty indifferent they seem much the same to me
   20:22:27	jive[m]:	I have not heard any complaints about the new server. At some point I would like to go over the bug list and see if we can get some of the issues fixed (things like images being duplicated each time they are cropped)
   20:23:02	robe2:	jive[m] hmm being duplicated wasn't aware, we can add to agenda next time
   20:23:16	robe2:	or talk on list
   20:23:26	robe2:	TemptorSent sorry for interruption
   20:25:03	TemptorSent:	Funtoo is the ongoing development effort by drobbins after splitting from Gentoo ~10yrs ago. The Funtoo organization operates a LXD containerized compute infrastructure, on which they are willing to donate some resources to the OSGeo project.
   20:25:22	strk[m]:	I'm comfortable with both Debian and Ubuntu, but I don't do much low level stuff
   20:26:02	TemptorSent:	The hardware this is running on is very similar to what we have proposed for purchase, so it's a good environment to get familiar with how the container environments work.
   20:26:35	strk[m]:	How many containers run on that hardware?
   20:27:19	TemptorSent:	Initially, my plan is to provide services for nextcloud, weblate, gitlab, and drone on the Funtoo containers, allowing us to beging testing those tools in the short term, and migrate as needed to our own hardware some or all services in the long term
   20:27:34	robe2:	strk[m] you're early
   20:27:46	wildintellect:	you can really run a ton of containers, it depends on load how many will run nicely
   20:27:55	TemptorSent:	I don't have the count myself, I'm sure palica can tell us later, or perhaps angry_vincent has it off hand?
   20:28:10	TemptorSent:	But the overhead per container is very, very minimal.
   20:28:53	strk[m]:	robe2: pizza is over, but I'm still sitting at the table
   20:29:02	TemptorSent:	Nesting is possible, including running VMs if configured to do so.
   20:29:05	robe2:	TemptorSent I love that idea :)
   20:29:47	strk[m]:	TemptorSent: do we get to test management of containers from the outside our just the inner?
   20:29:54	strk[m]:	Ah nesting!
   20:30:12	robe2:	dumb question how does the LXD container setup different from others we've talked about in past, in terms of being able to move container to our new hardware
   20:30:17	TemptorSent:	I'm working on writing the ebuilds for weblate and it's deps right now.
   20:30:27	robe2:	or would it be just a rebuild, which is fine too
   20:30:37	strk[m]:	So could we have an "osgeo" container in which we test running nested containers for the various services?
   20:30:47	TemptorSent:	LXD containers can be moved :)
   20:31:05	TemptorSent:	That's my current working hypothesis.
   20:31:40	TemptorSent:	We'll set up a container for each of the services under the primary container, allowing each to be managed or migrated independently.
   20:31:45	robe2:	ping bj0rn[m] you around know you were interested in experimenting with gitlab
   20:32:58	robe2:	and if we get a lot of use, we should donate some money to funtoo (for the cloudhosting)
   20:33:26	robe2:	somewhat comparable to how much advantage we are taking of it
   20:33:37	bj0rn[m]:	robe2: hi, yeah still somewhat interested :)
   20:33:38	TemptorSent:	Each service will have the minimal system needed to build and run it in it's container.
   20:34:14	TemptorSent:	robe2: I'm sure that would be appreciated if our needs grow greatly.
   20:34:37	robe2:	bj0rn[m] just in case you missed the conversation TemptorSent is talking about running these on Funtoo (funtoo would be providing the server and basic config)
   20:34:52	TemptorSent:	I believe Funtoo intends to provide reasonable resources as an in-kind donation to OSGeo.org.
   20:36:14	robe2:	bj0rn[m] so if we get this going on funtoo would you be willing to help out with the gitlab configuration stuff.
   20:36:46	bj0rn[m]:	TemptorSent: does this mean access to a gentoo-like box or should it be docker containers?
   20:36:49	TemptorSent:	For those interested, I'd be happy to give a crash-course on funtoo itself.
   20:37:22	robe2:	I'd be interested maybe we can do it in an impromptu next week sometime
   20:37:29	TemptorSent:	bj0rn[m] We will have root access within our containers to a funtoo environment (gentoo like, but with important difference)
   20:37:41	TemptorSent:	can do robe2.
   20:38:02	robe2:	TemptorSent maybe send out a note to sac list what time works best for you
   20:38:19	bj0rn[m]:	robe2: Sure, should try to migrate the existing experimental vm there. The valuable part is how to get it to auth with LDAP.
   20:38:21	strk[m]:	TemptorSent: so no apt-get there I guess?
   20:38:29	TemptorSent:	I'm building out the ebuilds for the services we will want to suite.
   20:38:40	robe2:	Even though OSGeo doesn't run on funtoo and may never, I always think we should try to support as many OS as we can
   20:38:44	strk[m]:	Or can we run a Debian in the container?
   20:38:44	TemptorSent:	strk[m] Nope, we use 'emerge'
   20:39:38	robe2:	it's pretty frustrating these days that seems only Ubuntu is tested, so I go on someone's system say BSD and Open source GIS doesn't run at all or as well (that's just my recent experience)
   20:39:41	TemptorSent:	I haven't personally experimented with debian in a funtoo container, -- systemd may cause issues for that configuration to play well without modification.
   20:39:50	robe2:	I think people on CentOS and others have similar problems
   20:40:15	strk[m]:	robe2: for software project sure, but for infra it just raises the amoubt of skills required to run it
   20:40:16	TemptorSent:	But nesting debian in ubuntu should be a non-issue.
   20:41:31	robe2:	strk[m] yah I agree not arguing with that but just saying for less mission critical stuff doesn't hurt to explore a bit
   20:41:33	strk[m]:	robe2: buildbot! (But Jenkins also allows for slaves IIRC)
   20:41:52	TemptorSent:	strk[m] - The infra is much less of an issue than the actual application configuration.
   20:42:01	robe2:	I know I know don't remind me -- anything to do with python scares me for some reason
   20:42:44	robe2:	yes indeed I have much more trouble getting my basic understandings of things right -- e.g. Jenkins on Windows runs about the same as it does on Debian
   20:43:05	robe2:	configuring is the same too so it's the same knowledge I have to know aside from dos vs bash scripts
   20:44:21	robe2:	same goes for PostgreSQL same crap you need to know whether you are running on Windows, FreeBSD, or some Linux (except when you start getting into ZFX configuring)
   20:44:27	TemptorSent:	Especially for webapps, customizing is much easier with ebuilds in my experience than with deb, which prevents falling back to building yourself.
   20:44:41	robe2:	which is someone else's probably anyway not a db programmer's problem
   20:45:17	wildintellect:	we pretty much avoid building ourselves if possible
   20:45:27	robe2:	TemptorSent but as they say freedom isn't free -- that's what scares strk and me
   20:45:29	wildintellect:	though the adhoc machine is the exception
   20:45:36	TemptorSent:	Once we have our configuration nailed down the way we like it, it's not terribly difficult to write a deb based on the ebuild.
   20:46:03	robe2:	in sense you don't want to have to build everything, but having some breathing room to build (outside of box) is nice
   20:46:44	robe2:	TemptorSent I think the concern is not so much the initial configuration but patch upgrades
   20:47:02	robe2:	we don't want to have to continually keep building those ourselves
   20:47:07	TemptorSent:	I can build everything custom without building everything by hand, which works well when a specific set of features is desired.
   20:47:53	TemptorSent:	Right, in most cases, it's just a matter of updating the version number for the ebuild and digesting the new tarball.
   20:48:13	robe2:	we'd have to decide which parts we really need custom and which are more hassle than it's worth
   20:48:32	TemptorSent:	cp blah-1.2.0.ebuild blah-1.2.1.ebuild && ebuild blah-1.2.1.ebuild digest
   20:48:43	robe2:	TemptorSent sorry I've been spoiled by apt install this that that2 apt upgrade
   20:48:56	robe2:	I've given up my freedom to save some key strokes
   20:49:26	TemptorSent:	Normal updates don't require any of that, only packages you've written custom.
   20:50:02	robe2:	okay as long as we can keep with the 95% normal updates 5% custom I'm happy
   20:50:32	robe2:	or at least that's my general feel at which the freedom of customness gets too biting
   20:50:32	TemptorSent:	Yep, it's just nice to be able to customize something deep in the chain and have it auto-rebuild everythign above it using your modification automatically.
   20:51:55	TemptorSent:	You don't run into unresolvable dependency-hell because you changed one package deep in the tree..
   20:52:43	TemptorSent:	And enabling/disabling features is a matter of adding or removing use flags to packages and rebuilding, not installing a different set of pacakages for each possible option.
   20:52:52	robe2:	TemptorSent I'll have to see it in action when we do the Funtoo ness.
   20:53:56	robe2:	one of those things you need to see, experience to believe -- but yes I have suffered dependency hell on debian and ubuntu, even without customizing my stuff. So I'm not against seeing a different way
   20:53:58	TemptorSent:	Indeed. I find it quite easy to deal with and develop on.
   20:54:28	robe2:	next topic git management of our infrastructure
   20:54:29	TemptorSent:	Each paradigm has it's place.
   20:55:25	robe2:	TemptorSent you should talk with dustymugs sometime in #postgis
   20:55:53	robe2:	he builds all his crap himself on slackware cause he doesn't trust anything he gets unless he configured it himself
   20:56:27	TemptorSent:	I'm not quite that bad, but I do like having control over my configuration.
   20:56:32	TemptorSent:	Okay, on to git :)
   20:56:36	robe2:	ping strk so have anything to say on git management of our architecture or you want Martin's input
   20:56:44	robe2:	if so maybe we can discuss more on list
   20:57:49	robe2:	okay strk must be eating pizza or bug sprinkled spaghetti again
   20:57:59	TemptorSent:	*lol*
   20:58:29	robe2:	next topic meeting time that Martin can actually make that we can ACTUALLY live with
   20:58:49	robe2:	from what I recall Martin's best time is something like Sunday morning
   20:59:32	robe2:	which I was willing to go with but was inconvenient from others -- should we just ask him on list to throw out 3 times that are convenient for him
   20:59:51	robe2:	and if we don't all scream at the ridiculousness of the times, we will pick one
   21:01:54	TemptorSent:	Okay, hopefully palica will be available shortly -- I'll be back in a bit.
   21:02:03	robe2:	Okay
   21:02:13	wildintellect:	robe2, that seems plausible
   21:02:42	robe2:	wildintellect okay so I'll send a note to list for Martin to pick 3 times that work best for him
   21:07:07	robe2:	Next topic Backup Machine
   21:07:14	robe2:	I didn't add that can anyone speak about it?
   21:11:27	strk:	home at last
   21:12:23	TemptorSent:	Backup Machine? What about it?
   21:12:32	strk:	these meetings w/out Martin don't sound very useful
   21:12:40	strk:	do you still want to know about git ?
   21:12:46	robe2:	TemptorSent it's on the agenda
   21:12:59	robe2:	strk sure tell us about git
   21:13:00	strk:	long story short: if anyone doesn't like it please setup and propose something different :)
   21:13:14	TemptorSent:	strk *LOL*
   21:13:17	strk:	I know it's a bad hack, but better than having no history at all
   21:13:17	robe2:	strk I was okay with just comitting to gitea
   21:13:42	TemptorSent:	As long as there are commit messages, it's probably fine.
   21:13:44	strk:	I'm also ok with just keeping locally (you'd "push" to Gitea, not "commit" - get up to speed with git nomenclature!)
   21:14:05	strk:	now, I can think of a somewhat better proposal
   21:14:11	robe2:	I did that for osgeo website changes I made. The wordpress part I started purging and replacing. I'll put a readme there cause I really will have configs with passwords when I am done
   21:14:19	strk:	but it takes time to implement and I'm not planning to do it myself, so not sure it is worth discussing :)
   21:14:27	robe2:	strk I like to push to gitea
   21:14:42	strk:	yeah, I'm happy Gitea is found useful :)
   21:14:45	robe2:	it drove me nuts having all these gits just local so I started committing them to gitea
   21:14:48	strk:	I've no problem with pushing to Gitea
   21:15:03	strk:	what Martin found out on one machine was a nested repository
   21:15:08	strk:	like /etc/.git and /etc/apache2/.git
   21:15:15	strk:	that's why he started "complaining"
   21:15:22	robe2:	oh
   21:15:27	strk:	he basically just asked for a fixed schema
   21:15:34	robe2:	yah I think I was just doing /etc/apache2
   21:15:41	robe2:	some things etc we never change
   21:16:08	strk:	I think this one is on "mail", so we get mail aliases, for example, under the generic one (/etc/.git)
   21:16:10	robe2:	and some things are really copied from others, like the ldap stuff I expect will be much the same on all machines, so seems pointless to commit
   21:16:19	robe2:	but apache2 is always different on each machine
   21:16:37	strk:	the reason I started "git init'ing" places was to leave a trace
   21:16:45	strk:	we could even just /.git :)
   21:16:48	TemptorSent:	Theoretically, an external git repo might make more sense than keeping multiple in /etc.
   21:16:57	strk:	right, and that's the "better way" :)
   21:17:06	strk:	better way could be using ansible playbooks
   21:17:13	strk:	a role for each service
   21:17:24	robe2:	yah but some like /etc/apache2 are really different for each server
   21:17:27	TemptorSent:	That might be a little much.
   21:17:32	strk:	and the whole osgeo playbook under a repository
   21:17:43	strk:	ansible is too much you think TemptorSent ?
   21:18:00	robe2:	my thinking is things that are specific to each server should be gited
   21:18:13	strk:	we're passed that part robe2 (I think)
   21:18:16	robe2:	everything else should be copied from a master git -- with a git export or whatever
   21:18:27	strk:	just git whatever you don't feel like changing witout a backup
   21:18:33	robe2:	strk passed what part?
   21:18:40	strk:	rather than leaving all backups around, put them under a git repo (that's my reasoning)
   21:18:55	strk:	less visible, in its nice dot-folder
   21:19:04	strk:	and with more metadata (if we handle to properly set GIT env variables)
   21:19:22	robe2:	strk yah I like that
   21:19:27	strk:	passed the part in which we think it's useful to keep history of changes
   21:19:34	strk:	we all agree on that I think ?
   21:19:41	robe2:	ansible I don't know about so can't comment
   21:19:49	strk:	what TemptorSent is saying (and it kind of makes sense) is: keep history off the machine
   21:19:55	strk:	if I understand correctly
   21:20:04	strk:	that'd mean moving "management" of machines outside of the machine
   21:20:24	robe2:	I'm not disciplined enough for that yet
   21:20:24	strk:	current approach is: I need to change something in service A, so I ssh into the machine running A and change something
   21:21:00	robe2:	strk my point is some history I don't care so much about
   21:21:18	TemptorSent:	History will be in a repo on the machine still, but not laying all over the place.
   21:21:21	strk:	a more "clean" approach is: I file a ticket against a repository holding the "state" of the service A, the change is committed there, the new state is deployed to the machine running A
   21:21:36	robe2:	say take the example of wordpress, I actually don't care that much about keeping update of some plugin, cause half the changes are in teh db anyway so that change is only half useful
   21:21:45	palica:	back
   21:21:52	TemptorSent:	WB palica.
   21:21:56	strk:	robe2: you are not forced to add everything in a git repo, so even if you have /.git (on the root) you can still only add what you care about
   21:22:00	robe2:	so plugins I would never edit, I'd just assume leave them out of gitea (but with a note, here is plugins we are using)
   21:22:09	robe2:	hi palica
   21:22:17	palica:	hi robe2 and others
   21:22:30	strk:	the only trouble with /.git is you'd probably want to add lots of things to .gitignore or `git status` would fill your screens
   21:22:37	robe2:	strk okay git on root works for me :)
   21:22:38	strk:	hi palica
   21:22:39	palica:	if I might chime in about config management
   21:22:42	TemptorSent:	palica - please check the scrollback -- questions RE how many containers the funtoo box is supporting on that harware IIRC
   21:22:45	palica:	take a look at salt
   21:22:50	palica:	saltstack
   21:23:03	strk:	saltstack, yay another technology, moar and moar !
   21:23:12	palica:	:)
   21:23:20	palica:	ok funtoo related
   21:23:29	palica:	current load on the servers is like 6 and 1
   21:23:46	palica:	about 100 containers running on both servers
   21:23:52	robe2:	strk you liked freedom the moar the betta
   21:24:00	strk:	saltstack website has text in light-gray over white, I can't trust them ! :)
   21:24:29	TemptorSent:	palica - Yeah, I'd say that's pretty light weight compared to what VM's would be eating :)
   21:24:33	strk:	6 and 1 is what ? load average ?
   21:24:45		* TemptorSent nods
   21:24:48	strk:	on 2 servers ?
   21:24:48	palica:	yes 15 minutes
   21:24:56	palica:	80 core HT
   21:25:09	palica:	128GB RAM
   21:25:46	palica:	zfs - L2ARC on ssd, ZIL (SLOG) on optane
   21:25:55	TemptorSent:	What's the mem usage look like?
   21:26:12	palica:	50% used
   21:26:18	strk:	80 core means 80 CPUs ?
   21:26:24	strk:	sorry for lame questions
   21:26:43	robe2:	80 cores isn't that like 20 cpus (4 cores / cpu)
   21:26:52	palica:	4 CPU each 10 core plus HT
   21:27:04	palica:	80 threads
   21:27:12	robe2:	ah
   21:27:31	strk:	so "6" as load average means only 6/80 of the available CPU has been used over the last 15 minutes ?
   21:27:46	robe2:	palica what kind of containers are you running currently on that? in terms of services
   21:28:07	strk:	tracsvn machine right now (for comparison): load average: 0.48, 0.52, 0.57
   21:28:10	palica:	funtoo supporters, funtoo web, jira, forums
   21:28:44	strk:	but that's in a VM, so not sure what the load average expresses
   21:28:45	palica:	funtoo merge_script to import changes from gentoo and add funtoo stuff
   21:28:54	TemptorSent:	To compare to our quoted HW, we'll have 16 cores, 32HT at 2.1GHz.
   21:29:59	strk:	tracsvn is 5 virtual CPUs if I read /proc/cpuinfo correctly, each at 2.5GHz
   21:30:13	palica:	Intel(R) Xeon(R) CPU E7- 4870 @ 2.40GHz
   21:30:29	TemptorSent:	Since we're not intending heavy compile loads, the 2.1GHz xeons should be fine.
   21:31:16	TemptorSent:	Our spec is 2x Xeon Silver 4110 @ 2.1GHz (85W TDP)
   21:31:17	strk:	osgeo6 (phisical machine) has 24cores with 1.2GHz each (I think)
   21:31:24	strk:	Intel(R) Xeon(R) CPU E5-2620 v3 @ 2.40GHz <--- this one is osgeo6 too
   21:33:34	palica:	funtoo containers just recently switched to LXD from OpenVZ
   21:33:46	palica:	LXD alone runs nicely
   21:33:46	TemptorSent:	palica - Can you elaborate a bit about Funtoo as an organization too please? :)
   21:34:06	palica:	nested LXD is what we have little experience so far
   21:34:19	palica:	so consider nested LXD beta
   21:34:45	palica:	organisation - best to ask drobbins about that
   21:34:59	robe2:	what's nested LXD?
   21:35:13	TemptorSent:	right - this is a good opportunity to test nesting and get any kinks worked out.
   21:35:20	palica:	basicaly you turn your container into a host
   21:35:29	TemptorSent:	robe2 - Yes, nested LXD.
   21:35:38	palica:	hosting other containers
   21:36:00	TemptorSent:	drobbins must be out for the day, I pinged him earlier.
   21:36:00	robe2:	ah so kinda like what strk was thinking? strk was that what you were thinking with an osgeo container?
   21:36:02	palica:	nested = LXD in LXD
   21:36:35	palica:	you would get container from funtoo and you could start any number of containers inside your container
   21:36:49	robe2:	palica but if you need to separate the containers out later you can or no?
   21:36:58	robe2:	I assume you can it's like any other container
   21:37:04	TemptorSent:	The idea would be to give Postgres, Weblate, Gitlab, Nextcloud, and Drone in their own sub-containers.
   21:37:07	palica:	yes
   21:37:33	palica:	you can even use lxd to move the containers around
   21:37:38	robe2:	okay sounds good to me and we would have root to this main container is my understanding right?
   21:37:38	TemptorSent:	That way migration of any one service has no impact on the others as long as the networking is right.
   21:37:44	palica:	all you need is another lxd instance
   21:37:49	palica:	you can do something like
   21:38:02	palica:	lxc mv container-1 remote:container-1
   21:38:28	palica:	or you can just make a archive
   21:38:37	palica:	and import it on the other machine
   21:38:42	robe2:	sounds easy
   21:38:55	robe2:	So how do we get started
   21:39:03	TemptorSent:	Much easier than managing VMs IMHO.
   21:39:11	palica:	https://linuxcontainers.org/lxd/try-it/
   21:39:12	sigabrt:	Title: Linux Containers - LXD - Try it online (at linuxcontainers.org)
   21:39:35	palica:	here you can test the basic functions of lxd
   21:39:46	robe2:	oh I forgot so do containers maintain all state stuff so not like Docker where things are lost on shutdown
   21:39:51	TemptorSent:	palica - ssh keys needed from our end, right?
   21:40:06	palica:	for the container?
   21:40:08	palica:	yes
   21:40:11	TemptorSent:	Nope, fully persistent just like a VM would be.
   21:40:32	palica:	so the container share same kernel with the host
   21:40:56	palica:	and you can think about containers as chroot on steroids
   21:41:08	robe2:	okay that's good. One of the things I really hated about docker
   21:41:18	palica:	with cgroup management (resource management) and namespaces (isolation)
   21:41:29	robe2:	some things don't want to bother with the hassle of running on two, some we might
   21:41:53	palica:	what is your load on the servers?
   21:42:23	palica:	traffic?
   21:43:20	TemptorSent:	Primary servers get moderate traffic -- reports around here somewhere, lemme find a URL.
   21:43:57	TemptorSent:	The funtoo services will be used for services not currently provisioned that have been requested and need configuarion, evaluation, and testing.
   21:44:56	TemptorSent:	High network, low compute services will probably migrate to the OSGeo HW once purchased and provisioned, which will be COLO at OSUOSL.
   21:45:35	TemptorSent:	The higher burst compute loads (compiling, etc.) will probably want to remain on the funtoo servers.
   21:47:20	robe2:	after party meeting started
   21:48:09	palica:	ok, anything else?
   21:48:25	TemptorSent:	http://webextra.osgeo.osuosl.org/munin/osgeo.org for munin stats palica
   21:48:26	sigabrt:	Title: Munin :: osgeo.org (at webextra.osgeo.osuosl.org)
   21:49:41	robe2:	palica thanks for the introduction I'll start playing with try it to get a feel for how all this works.
   21:49:43	markusN:	Next week is OSGeo Code Sprint here in Bonn
   21:49:53	markusN:	anyone there from you?
   21:50:01	TemptorSent:	Thank you palica.
   21:50:19	palica:	np
   21:50:41	markusN:	besides that: Martin wanted to help me to deploy a Wordpress docker + mysql docker on osgeo6 so that we can finally renew the GRASS GIS CMS
   21:50:54	markusN:	any problems in doing so ?
   21:51:00	strk:	robe2: yes, nested containers is what both me and TemptorSent were thinking
   21:52:02	TemptorSent:	It would probably be the easiest to manage.
   21:52:02	palica:	just a footnote: if you could afford to support funtoo in any way that would be greatly appreciated
   21:52:30	robe2:	palica yap we talked about that and are planning to once we start going
   21:52:31	strk:	fully persistent is an interesting difference from docker
   21:52:36	TemptorSent:	Oh, logo exchange would be wise too :)
   21:52:43		* strk is reading backlog, sorry for out-of-sync noise
   21:52:58	robe2:	We can at the very least do logo exchange I'm sure but wanted to do a bit more than that
   21:53:50	strk:	lovely terminal on linuxcontainers.org/lxd/try-it, I like it, is that html5 ?
   21:54:01	robe2:	markusN not sure details of osgeo6
   21:54:10	TemptorSent:	As an aside, I'm working on bringing the full OSGeo stack up to first-class status on Funtoo.
   21:54:14	palica:	I think that even the container prices are pretty reasonable https://www.funtoo.org/Funtoo_Containers
   21:54:15	sigabrt:	Title: Funtoo Containers - Funtoo (at www.funtoo.org)
   21:54:36	robe2:	do we have mysql runnong on osgeo6. We have wordpress already running on web18a for osgeo -- wondering if it would be easier to just setup another site on that box
   21:55:06	robe2:	markusN which PHP version are you currently running GRASS GIS CMS on?
   21:55:10	palica:	ok, guys good night
   21:55:21	TemptorSent:	Yes, those prices are somewhat insanely low for the resources :)
   21:55:30	TemptorSent:	Goodnight palica, and thanks again.
   21:55:35	palica:	if yoe need something don't hessitate to come to #funtoo or #funtoo-dev
   21:55:43	palica:	or support@funtoo.org
   21:55:59	markusN:	robe2: we want to entirely redo the CMS
   21:56:11	markusN:	move from CMSMS to WP (for now)
   21:56:44	robe2:	markusN oh so you are not running wordpress currently?
   21:56:48	markusN:	probably just setting up another site may be fine
   21:56:50	markusN:	no
   21:57:05	palica:	no problem, bye
   21:57:10	markusN:	anyone able to set up another site? then we could work on content next week at the sprint
   21:57:14		* TemptorSent waves to palica.
   21:57:21		* markusN waves
   21:57:26	robe2:	I could setup another site, right now it's a 4GB and I think if we need to we can increase the power on it (it's a cloud server we are hosting from OSUOSL)
   21:57:28		* palica waves back :)
   21:58:13	robe2:	markusN you have your request ticketed already? my only concern with docker is how it manages the data, but Martin would know more about that
   21:58:31		* TemptorSent mumbles something about containers would be perfect for the above task.
   21:58:54	robe2:	and since we already have MariaDB, PHP7 etc. already setup on new server (I have staging.www.osgeo.org and www.osgeo.org , would be trivial to put up another wordpress)
   21:59:42	markusN:	Martin seems to be busy, likely our docker stuff is not prio 1
   21:59:50	TemptorSent:	Yeah, I haven't played with WP's guts in a while, it used to be a little bit painful in multitennent use due to conflicting naming, but I suspect that's been fixed for a while now.
   21:59:53	robe2:	TemptorSent yah containers would be perfect for above too but since we haven't experimented with that and GRASS I think needs to be very robust, seemed safer to er on what we've already got going and tested
   22:00:08	markusN:	fine: what to request precisely in the ticket to avoid confusion?
   22:00:30	robe2:	TemptorSent I don't bother with multitenanting - I'd just setup another database and anotehr apache website on same box
   22:00:42	TemptorSent:	Yeh, I know -- I'm just looking at use cases for containers vs vms on the new hw and haven't seen many cases where a VM would be necessary yet.
   22:00:50	robe2:	osgeo has all these plugin crap that GRASS probably doesn't need and would get in the way
   22:01:02	markusN:	our current CMS is too old and not responsive, looks horrible as of 2018 (ok even 2016) - time to fix that
   22:01:09	TemptorSent:	robe2 - Ahh, yeah - that's a better option then.
   22:01:21	markusN:	we would be fine with a simple WP
   22:01:29	TemptorSent:	I'm not a big WP fan, but it's easy.
   22:01:42	strk:	TemptorSent: on the "try-it" URL, I see in the lits of ubuntu: images there are multiple architectures
   22:02:02	strk:	does it mean a single machine (say x86_64) can run containers with a different architecture ?
   22:02:08	strk:	as in a real VM ?
   22:02:17	markusN:	I'm not a big WP fa either but we need to be pragmatic here
   22:02:18	robe2:	TemptorSent yah I'm not a big WP fan either but hey all my clients are into it so got growing experience setting them up and walking away :)
   22:02:25	strk:	or does it only mean that images are available for runnig from the different CPUs ?
   22:02:39		* strk is not a fan of WP either (althought running one)
   22:02:51	strk:	lack of PostgreSQL backend is unjustified nowadays :)
   22:02:59	robe2:	I wish they had better support for PostgreSQL
   22:03:29	robe2:	strk you beat me to that comment yah my thoughts that's one of the reasons I don't use it
   22:03:43	strk:	markusN: had you considered a git-based setup ?
   22:03:48	robe2:	in theory it works with postgres but I hear not well
   22:03:55	strk:	or do you really need things like comments gathering ?
   22:04:02	markusN:	we did ascii stuff for 10+ years, then moved to CMS
   22:04:04	strk:	git-based would be generated once (not upon receiving requests)
   22:04:13	TemptorSent:	Likewise -- if it doesn't support PG, it better have a damn good excuse.
   22:04:22	strk:	markusN: is that to have a web UI to modify it ?
   22:04:27	markusN:	just there is nobody who would maintain it (realistically)
   22:04:29	strk:	and WYSIWYG ?
   22:04:32	markusN:	yes
   22:04:38	strk:	ok, makes sense
   22:04:43	markusN:	otherwise I edit all alone for another 10 years :p
   22:04:54	robe2:	markusN you have this request ticketed already?
   22:04:59	markusN:	no
   22:05:08	markusN:	just request a WP instance?
   22:05:17	robe2:	yes
   22:05:29	markusN:	WIP
   22:05:31	TemptorSent:	*lol* I still write my html from scratch or write the code to generate it.
   22:05:40	robe2:	You can even write I promised to set it up on web18a which is already all configured for wordpress
   22:05:50	markusN:	perfect :)
   22:05:55	strk:	TemptorSent: I also write my html (strk.kbt.io) - but one day I decided I'd setup a blog, which I did with WP
   22:06:05	strk:	advantage ? being published to planet.osgeo.org
   22:06:13	robe2:	I'll try to get to it this weekend and make your account an admin (I'll install the LDAP authentication plugin)
   22:06:15	strk:	I was too lazy to manually write an rss
   22:06:31	TemptorSent:	strk - Yes, VMs can run for various archs with containers inside THOSE
   22:06:51	strk:	ah, so you'd have container -> VM -> container ?
   22:07:09	strk:	osgeo(container) --> arm(vm) -> container(build-slave-for-drone)
   22:07:11	strk:	for example ?
   22:07:41	TemptorSent:	strk Possibly, although the vm may require privledged containers in some cases.
   22:07:49	markusN:	great, thank you!
   22:07:55	strk:	robe2: ready for Drone matrix testing PostGIS against multiple archs ? :)
   22:08:12	TemptorSent:	That's entirely possible :)
   22:08:45	robe2:	strk oh yah that would be great
   22:09:03	strk:	damn ^w, I closed the try-it tab accidentally
   22:09:22	TemptorSent:	*LOL* Hate that.
   22:09:37	strk:	like a power-cycle for that containers
   22:10:05	strk:	I love that terminal btw, really nice
   22:10:06	markusN:	https://trac.osgeo.org/osgeo/ticket/2131
   22:10:07	sigabrt:	Title: #2131 (New Wordpress instance for GRASS GIS project) – OSGeo (at trac.osgeo.org)
   22:10:16	markusN:	thanks again for your support
   22:10:44	strk:	damn, I did it again !
   22:11:00	strk:	^w is _needed_ for me in a terminal, it can't just close the damn tab, anyone knows how to disable that thing ?
   22:11:03	strk:	(firefox)
   22:11:23	robe2:	markusN put a note on their what should be the domain name
   22:11:25	strk:	actually, I'll go to sleep, have fun !
   22:11:43	robe2:	I presume it would be a staging domain name so you have time to move stuff over
   22:11:44	TemptorSent:	G'night strk, and Keybindings?
   22:12:15	markusN:	robe2: dunno, something grass_temp.osgeo.org ?
   22:12:27	robe2:	sure that'll work
   22:12:30	markusN:	eventually it will become grass.osgeo.org after our job is done
   22:13:48	TemptorSent:	Anyone wanting to play with on their own machines, feel free to pop over to #funtoo and ask for help any time -- we have a very active small community over there.