Git and markdown and CD0

Git and markdown and CD0

Sir Russ the Just prevails upon me: I'm increasingly using Git for projects.
What's so great about Git?

 Git

Not personally stuck on any one content manglement system. (The acronym CMS is a whole other story.) But Git seems to have advantages for collaborative coding. Hearing the hype, my radar perked up when the GitHub guys offered a free intro class (now five years ago). 

At work we lean primarily on Subversion. But Git looms large. My teammate, the Nomad, gave me his take on the matter. For him, Git works better when you need to branch, test, merge, and commit remotely. (So maybe that helps nomadic life.) The corporately sponsored GitHub server is available to anyone with credentials, not merely our division (and with no special dispensation from our over-worked local devops team). Normalized access is a major plus. Anyone on my team can use it too.

Markdown

Speaking of remote operation, at work we also use Confluence. This popular documentation tool must be incredibly chatty. I haven't analyzed the traffic. All I can say is that (for me, 100% remote) it's ssslllooowww with a capital SSS. (Evidently not slow for those at HQ.) If Confluence pages under the covers were plaintext (HTML or other markup), we could RSYNC to/from the server and make changes easily. But Confluence wants to be interactive by way of your web browser. (Something about hammers and nails and "everybody has Internet Exploder", so what were Confluence designers thinking?)

I don't know what I don't know about Confluence. Gentle reader, please be gentle if you have myriad clues that I lack about the wonders of Confluence and how it can be made to work better. Maybe you'll say, "yes, Rick, you can RSYNC from/to Confluence".
Meanwhile, enter Markdown.

Another nudge from Russ is to get me free from the talons of Google, specifically their documentation service. (Let's not talk about the fact that this post has its genesis on another Google property.) Russ says I should use markdown for project docs. I offered no resistance: markdown is plaintext, it's portable, it's simple. At work, markdown docs on the corporate GitHub server are a refreshing alternative to Confluence pages. For hobbies, I find myself converting Google docs to markdown as time allows.

CD0

All of this comes to a head because NORD has a "GitHub project" presence which is now getting more TLC. This is where I'm burning most of that doco conversion time, moving files from Google Drive to GitHub. But being GitHub, this is also where the non-doc files have been mirroed for a long time. The effect is a convergence of doc files and operational files into a common space. Good!

What's lacking is a spreadsheet, or something that looks/tastes/smells like a spreadsheet, which can serve both for documentation and operation. As NORD evolved, it helped to have a place to publicly list what packages are current. (NORD eschews formal package management, which would include inventory, for the sake of sticking closer to the individual package sources.) Using the common spreadsheet as a reference, one can get a local NORD installation up to date programmatically. Markdown does support tables, so that might be a workable alternative to a proper spreadsheet.

What's CD0?

NORD is comprised of a core operating system with "optional" packages drawn in on demand. The optional packages are built with Chicory so they can reside anywhere, but as a starting point there are collections under /opt named CD1 through CD5. (Maybe more later.) The naming is intentional because eash collection could literally reside on a CD or DVD or in an ISO-9660 image. In any case, separate filesystems is ovten reasonable.

CD0 is a special case. The packages in CD0 are the Chicory build of core packages (normally built with CSCRATCH). CD0 doesn't have the entire core package set, but by design has enough packages for "rootstrapping". Supplement packages from CD0 with those from CD1, CD2, CD3, and a fresh NORD core system root can be derived. Once the new NORD core system has been built, CD0 is no longer needed.

Summary, only tangential to Git and Markdown, the current CD0 set is ...

• autoconf-2.69
• automake-1.15
• bash-4.4
• bison-3.0.4
• bzip2-1.0.6
• coreutils-8.27
• curl-7.54.1
• dash-0.5.9.1
• diffutils-3.6
• file-5.31
• findutils-4.6.0
• flex-2.5.39
• gawk-4.1.4
• gettext-0.19.8
• grep-3.1
• gzip-1.8
• libtool-2.4.6
• m4-1.4.18
• make-3.79.1
• nano-2.8.6
• ncurses-6.0
• patch-2.7.5
• pdksh-5.2.14
• readline-7.0
• rsync-3.1.2
• screen-4.6.1
• sed-4.4
• sharutils-4.15.2
• tar-1.22
• texinfo-6.4
• wget-1.19
• which-2.21
• xz-5.2.3
• zlib-1.2.11

That's 34 packages. Some of them are behind current releases for one reason or another. With Chicory, both old and new can co-exist on the system, but this set is intended for rootstrapping. Now to put the list into table form.

-- R; <><

XMITMSG for POSIX

XMITMSG for POSIX

I grew up using VM/CMS and wrote a few programs for that platform. While most of my world is Unixy, there are some things in *CMS just begging to be ported. One is 'XMITMSG' (the command) and "APPLMSG" (the assembler macro). Together they form the interface to the message handler used by most well-developed CMS programs.

Just Say It

'echo', Say, printf(), ... so many ways to write a line of output. And since we all want to keep it simple, who would ask for the complexity of a "message handler" just to display verbiage? Years ago, I wrote a certain client program that was really well received. That was fun! Someone actually liked my program. Got lots of feedback and feature requests. One requested change kinda put me off, "Use XMITMSG.". What the heck is XMITMSG? Why would I want to use it? "You can change the message repository for other languages.". Okay, now that is interesting, and cool, but sounds like a lot of work. "It's easy!", and it turned out it was.

I submitted to the desires of my users, changed out the "Say" statements, replacing them with calls to the 'XMITMSG' program. Had to put all the verbal content into a message repository, but it worked. I was hooked! Not too long after, someone contributed a message repository for Portuguese. Wow. So my client was the only internationalized program of its kind.

More Strucured

It's not just about internationalization. Using the message handler meant that human readable output was better managed. Forcing myself to use 'XMITMSG' meant I had to think just a little bit harder about what each print operation was actually doing. I'm naturally un-structured, which seems to be contrary for techies, especially coders, so the utility was a real help for me in development. On other platforms, I really miss that CMS message handler.

I cobbled up a crude 'xmitmsg' for Unix. It's still out there, but last time I tinkered with it there were compiler issues. (C has changed, and so have I. A re-write was in order. See below.) 

Beyond organization, going through a central function also means you can direct traffic more effectively. What if you want something logged instead of displayed? What if you want certain output muted? These things are done for free with a proper display handler.

But We can Hash and Replace

Later, I was on a team developing a multi-platform product. Part of it ran on CMS. On the VM side, we had this built-in function. It would be trivial to do it on AIX and Solaris and Linux. (I had already done it and could do it better in a formal context.)

The Unix guys would not go for it. One in particular, a brilliant engineer named Mike, conveyed to me that it just didn't make sense from his perspective. (Almost like a hygiene thing and calling out messages by number was somehow unclean.) Instead they wanted to hash the messages and do string replacement. Not sure what they were using: sounded sorta like GetText but probably a fore-runner. I lost that battle. The product shipped. Move along. Nothing to see here.

And Then Came FUZIX

More than two years ago, I added an XMITMSG work-alike project to my GitHub space. Nuthin happnin; no time. But I've been prompted to get back into GitHub.

One of the projects I follow (for certain values of "follow"; who has time??) is FUZIX. Small is beautiful. FUZIX is a Unix like system for 8-bit processors. Remember the 6502? Yeah, that one. It's an ongoing effort with lots of contributors doing 'make'. One of the contributors posted the output of a failing build which contained this:

fallo en las instrucciones para el objetivo 'stty.rel'

Which in English is:

recipe for target 'stty.rel' failed

This is a perfect candidate for XMITMSG.
This is the kind of thing the message handle project is intended to address.

XMITMSGX

The project is called "xmitmsgx", note the trailing "x", XMITMSG for POSIX.
It works.
It needs work.

https://github.com/trothr/xmitmsgx/

Check it out. Maybe do a pull request and contribute some code.

The initial cut is compatible with CMS 'XMITMSG', same message source syntax, same message code format, stuff like that. But let's think outside the box and realize that other input and output are doable. Meanwhile, it helps to have a known working model to compare against.

-- R; <><

*CMS is the "Conversational Monitor System" component of z/VM and previous incarnations of that operating system. The acronym is so over-used that most readers probably think of something else when they see it. In this post, I'm explicitly talking about VM/CMS even if I leave off the "VM/" prefix. Capisce?