tag:blogger.com,1999:blog-25065329163297435182024-03-13T05:28:04.490-07:00Sir SantaRick Trothhttp://www.blogger.com/profile/03475933977328026233noreply@blogger.comBlogger35125tag:blogger.com,1999:blog-2506532916329743518.post-19546519116621223992017-12-30T11:40:00.000-08:002017-12-30T11:40:46.955-08:00Git and markdown and CD0<b>Git and markdown and CD0</b><br />
<br />
Sir Russ the Just prevails upon me: I'm increasingly using Git for projects.<br />
What's so great about Git?<br />
<br />
<b>Git</b> <br />
<br />
Not personally stuck on any one content manglement system. (The acronym CMS is a whole other story.) But Git seems to have advantages for collaborative coding. Hearing the hype, my radar perked up when the GitHub guys offered a free intro class (now five years ago). <br />
<br />
At work we lean primarily on Subversion. But Git looms large. My teammate, the Nomad, gave me his take on the matter. For him, Git works better when you need to branch, test, merge, and commit remotely. (So maybe that helps nomadic life.) The corporately sponsored GitHub server is available to anyone with credentials, not merely our division (and with no special dispensation from our over-worked local devops team). Normalized access is a major plus. Anyone on <i>my</i> team can use it too. <br />
<br />
<b>Markdown</b><br />
<br />
Speaking of remote operation, at work we also use Confluence. This popular documentation tool must be incredibly chatty. I haven't analyzed the traffic. All I can say is that (for me, 100% remote) it's ssslllooowww with a capital SSS. (Evidently not slow for those at HQ.) If Confluence pages under the covers were plaintext (HTML or other markup), we could RSYNC to/from the server and make changes easily. But Confluence wants to be interactive by way of your web browser. (Something about hammers and nails and "everybody has Internet Exploder", so what were Confluence designers thinking?)<br />
<br />
I don't know what I don't know about Confluence. Gentle reader, please be gentle if you have myriad clues that I lack about the wonders of Confluence and how it can be made to work better. Maybe you'll say, "yes, Rick, you can RSYNC from/to Confluence". <br />Meanwhile, enter Markdown.<br />
<br />
Another nudge from Russ is to get me free from the talons of Google, specifically their documentation service. (Let's not talk about the fact that this post has its genesis on another Google property.) Russ says I should use markdown for project docs. I offered no resistance: markdown is plaintext, it's portable, it's simple. At work, markdown docs on the corporate GitHub server are a refreshing alternative to Confluence pages. For hobbies, I find myself converting Google docs to markdown as time allows.<br />
<br />
<b>CD0</b><br />
<br />
All of this comes to a head because <a href="http://www.casita.net/pub/nord/" target="_blank">NORD</a> has a "GitHub project" presence which is now getting more TLC. This is where I'm burning most of that doco conversion time, moving files from Google Drive to GitHub. But being GitHub, this is also where the non-doc files have been mirroed for a long time. The effect is a convergence of doc files and operational files into a common space. Good!<br />
<br />
What's lacking is a spreadsheet, or something that looks/tastes/smells like a spreadsheet, which can serve both for documentation and operation. As NORD evolved, it helped to have a place to publicly list what packages are current. (NORD eschews formal package management, which would include inventory, for the sake of sticking closer to the individual package sources.) Using the common spreadsheet as a reference, one can get a local NORD installation up to date programmatically. Markdown does support tables, so that might be a workable alternative to a proper spreadsheet.<br />
<br />
<b>What's CD0? </b><br />
<br />
NORD is comprised of a core operating system with "optional" packages drawn in on demand. The optional packages are built with Chicory so they can reside anywhere, but as a starting point there are collections under <span style="font-family: "Courier New",Courier,monospace;">/opt</span> named <span style="font-family: "Courier New",Courier,monospace;">CD1</span> through <span style="font-family: "Courier New",Courier,monospace;">CD5</span>. (Maybe more later.) The naming is intentional because eash collection could literally reside on a CD or DVD or in an ISO-9660 image. In any case, separate filesystems is ovten reasonable.<br />
<br />
CD0 is a special case. The packages in CD0 are the Chicory build of core packages (normally built with CSCRATCH). CD0 doesn't have the entire core package set, but by design has enough packages for "rootstrapping". Supplement packages from CD0 with those from CD1, CD2, CD3, and a fresh NORD core system root can be derived. Once the new NORD core system has been built, CD0 is no longer needed.<br />
<br />
Summary, only tangential to Git and Markdown, the current CD0 set is ...<br />
<br />
<ul>
<li>autoconf-2.69</li>
<li>automake-1.15</li>
<li>bash-4.4</li>
<li>bison-3.0.4</li>
<li>bzip2-1.0.6</li>
<li>coreutils-8.27</li>
<li>curl-7.54.1</li>
<li>dash-0.5.9.1</li>
<li>diffutils-3.6</li>
<li>file-5.31</li>
<li>findutils-4.6.0</li>
<li>flex-2.5.39</li>
<li>gawk-4.1.4</li>
<li>gettext-0.19.8</li>
<li>grep-3.1</li>
<li>gzip-1.8</li>
<li>libtool-2.4.6</li>
<li>m4-1.4.18</li>
<li>make-3.79.1</li>
<li>nano-2.8.6</li>
<li>ncurses-6.0</li>
<li>patch-2.7.5</li>
<li>pdksh-5.2.14</li>
<li>readline-7.0</li>
<li>rsync-3.1.2</li>
<li>screen-4.6.1</li>
<li>sed-4.4</li>
<li>sharutils-4.15.2</li>
<li>tar-1.22</li>
<li>texinfo-6.4</li>
<li>wget-1.19</li>
<li>which-2.21</li>
<li>xz-5.2.3</li>
<li>zlib-1.2.11</li>
</ul>
<br />
That's 34 packages. Some of them are behind current releases for one reason or another. With Chicory, both old and new can co-exist on the system, but this set is intended for rootstrapping. Now to put the list into table form.<br />
<br />
-- R; <><<br />
<br />
<br />
<br />
<br />
Rick Trothhttp://www.blogger.com/profile/03475933977328026233noreply@blogger.com1tag:blogger.com,1999:blog-2506532916329743518.post-62181174038953426362017-12-03T21:42:00.000-08:002017-12-03T21:42:06.356-08:00XMITMSG for POSIX<b>XMITMSG for POSIX</b><br />
<br />
I grew up using VM/CMS and wrote a few programs for that platform. While most of my world is Unixy, there are some things in *CMS just begging to be ported. One is 'XMITMSG' (the command) and "APPLMSG" (the assembler macro). Together they form the interface to the message handler used by most well-developed CMS programs.<br />
<br />
<b>Just Say It</b><br />
<br />
'<span style="font-family: "courier new" , "courier" , monospace;">echo</span>', Say, <span style="font-family: "courier new" , "courier" , monospace;">printf()</span>, ... so many ways to write a line of output. And since we all want to keep it simple, who would ask for the complexity of a "message handler" just to display verbiage? Years ago, I wrote a certain client program that was really well received. That was fun! Someone actually <i>liked</i> my program. Got lots of feedback and feature requests. One requested change kinda put me off, "Use XMITMSG.". What the heck is XMITMSG? Why would I want to use it? "You can change the message repository for other languages.". Okay, now that <i>is</i> interesting, and cool, but sounds like a lot of work. "It's easy!", and it turned out it was.<br />
<br />
I submitted to the desires of my users, changed out the "Say" statements, replacing them with calls to the 'XMITMSG' program. Had to put all the verbal content into a message repository, but it worked. I was hooked! Not too long after, someone contributed a message repository for Portuguese. Wow. So my client was the only internationalized program of its kind.<br />
<br />
<b>More Strucured</b><br />
<br />
It's not just about internationalization. Using the message handler meant that human readable output was better managed. Forcing myself to use 'XMITMSG' meant I had to think just a little bit harder about what each print operation was actually doing. I'm naturally un-structured, which seems to be contrary for techies, especially coders, so the utility was a real help for me in development. On other platforms, I really miss that CMS message handler.<br />
<br />
I cobbled up a crude '<span style="font-family: "courier new" , "courier" , monospace;">xmitmsg</span>' for Unix. It's still out there, but last time I tinkered with it there were compiler issues. (C has changed, and so have I. A re-write was in order. See below.) <br />
<br />
Beyond organization, going through a central function also means you can direct traffic more effectively. What if you want something logged instead of displayed? What if you want certain output muted? These things are done for free with a proper display handler. <br />
<br />
<b>But We can Hash and Replace</b><br />
<br />
Later, I was on a team developing a multi-platform product. Part of it ran on CMS. On the VM side, we had this built-in function. It would be trivial to do it on AIX and Solaris and Linux. (I had already done it and could do it better in a formal context.) <br />
<br />
The Unix guys would not go for it. One in particular, a brilliant engineer named Mike, conveyed to me that it just didn't make sense from his perspective. (Almost like a hygiene thing and calling out messages by number was somehow unclean.) Instead they wanted to hash the messages and do string replacement. Not sure what they were using: sounded sorta like GetText but probably a fore-runner. I lost that battle. The product shipped. Move along. Nothing to see here.<br />
<br />
<b>And Then Came FUZIX</b><br />
<br />
More than two years ago, I added an XMITMSG work-alike project to my GitHub space. Nuthin happnin; no time. But I've been prompted to get back into GitHub.<br />
<br />
One of the projects I follow (for certain values of "follow"; who has time??) is FUZIX. Small is beautiful. FUZIX is a Unix like system for 8-bit processors. Remember the 6502? Yeah, that one. It's an ongoing effort with lots of contributors doing '<span style="font-family: "courier new" , "courier" , monospace;">make</span>'. One of the contributors posted the output of a failing build which contained this:<br />
<br />
<div style="text-align: center;">
fallo en las instrucciones para el objetivo 'stty.rel'</div>
<br />
Which in English is: <br />
<br />
<div style="text-align: center;">
recipe for target 'stty.rel' failed</div>
<br />
This is a perfect candidate for XMITMSG.<br />
This is the kind of thing the message handle project is intended to address.<br />
<br />
<b>XMITMSGX</b><br />
<br />
The project is called "xmitmsgx", note the trailing "x", XMITMSG for POSIX.<br />
It works.<br />
It needs work. <br />
<br />
<div style="text-align: center;">
<span style="font-family: "courier new" , "courier" , monospace;">https://github.com/trothr/xmitmsgx/</span></div>
<br />
Check it out. Maybe do a pull request and contribute some code.<br />
<br />
The initial cut is compatible with CMS 'XMITMSG', same message source syntax, same message code format, stuff like that. But let's think outside the box and realize that other input and output are doable. Meanwhile, it helps to have a known working model to compare against. <br />
<br />
-- R; <><<br />
<br />
*CMS is the "Conversational Monitor System" component of z/VM and
previous incarnations of that operating system. The acronym is so
over-used that most readers probably think of something else when they
see it. In this post, I'm explicitly talking about VM/CMS even if I
leave off the "VM/" prefix. Capisce? <br />
<br />
<br />
<br />Rick Trothhttp://www.blogger.com/profile/03475933977328026233noreply@blogger.com0tag:blogger.com,1999:blog-2506532916329743518.post-42275715928483942062017-11-05T17:42:00.001-08:002017-11-05T17:42:10.168-08:00Call it Lennarx<b>Lennarx Assimilation</b><br />
<br />
<br />
I took the mic at a certain conference, Q&A after the speaker made his pitch, shared my concerns about the topic. There were probably 300 people in that session. The speaker was obviously pro where I was con. Most of the audience were supporters, enthusiastic just to hear. No one spoke up for my side. (i.e., to second my statement or raise their own questions or worries) Possibly not one else was <i>on</i> "my side". <br />
I felt alone. <br />
<br />
<br />
More recently, my colleague Phil said "it's good to be contrarian". That's comforting, if not effective. I'd really like to bring actual change and not merely whine about the badness.My wife would surely appreciate if I were less contrarian in general. <br />
<br />
<br />
<b>Linux and SystemD </b><br />
<br />
<br />
Speaking of whining, there's lots of that over SystemD. Time and again I hear from people who hate it. But it's not going away. The criticisms are legimitate, even if opinion. For example, SystemD assimilating the logging function is gross overreach, but not technically insurmountable. I can say I dislike the fact that logging is rolled into SystemD but can't prove that it doesn't work. So I (and others, I know they're out there) sit here under the rule of Lennart. <br />
<br />
<br />
Then my friend Russ sent this ... <br />
<br />
<br />
<br />
<div style="text-align: center;">
<span style="font-family: "courier new" , "courier" , monospace;">http://lkml.iu.edu//hypermail/linux/kernel/1408.1/02496.html</span></div>
<br />
<br />
I don't know Christopher Barry, but I agree with everything he said. (He said it better than I could have and enumerated more facts than I would have.) <br />
<br />
<br />
<b>Call it Lennarx</b><br />
<br />
<br />
If the kernel started by Linus is Linux, then the system daemon started by Lennart is Lennarx. Why not? <br />
<br />
<br />
The pro-SystemD crowd would just as soon see the rest of us get over it, shut up, and accept "this is Linux". I've never understood (or never cared about) the benefits. They've never understood (or never cared about) the costs of it, what the rest of us lost. And what did we lose? Simplicity, for one thing. This is not a "who moved my cheese?" story. <br />
<br />
For me, there's also the loss of interoperability. (SysV INIT is generally compatible with other Unix systems.) And I was told that one advantage of SystemD is faster boot times.Hasn't happened. Rebooting takes just as long, sometimes longer. But think about it ... faster boot time? WHY do we want to improve something we don't really want to DO so often? <br />
<br />
<br />
<b>There is Hope</b><br />
<br />
<br />
Though we're often accused of tilting at windmills, there is real hope. The majority will no doubt continue with SystemD but the alternatives are out there. Hearing from people like Christopher Barry prove that we're not alone. Most of us also use the BSDs. And that's kinda the point: it's more about Unix and FOSS built for that API. <br />
<br />
<br />
FOSS runs everywhere. FOSS fans have always had to fight FUD. This latest borg battle has made major inroads. It has assimilated some of our friends. But the remnant remains. <br />
<br />
<br />
-- R; <><<br />
<br />
<br />
<br />Rick Trothhttp://www.blogger.com/profile/03475933977328026233noreply@blogger.com0tag:blogger.com,1999:blog-2506532916329743518.post-79814462343965699162017-10-30T06:53:00.000-07:002017-10-30T06:53:55.394-07:00NORD Updates for 2017<span style="font-size: large;"><b>NORD Updates for 2017</b></span><br />
<br />
<br />
Here's the latest status and package updates for NORD.<br />
<br />
<br />
NORD is in production use for both internal and external service at casita.net, and is presently reliable and stable. But it's smart to keep current.<br />
<br />
<br />
<b>NORD Updates - core operating system</b><br />
<br />
<br />
Most of the NORD systems at casita.net use a shared operating system. For virtual machines, the OS resides on a shared virtual disk and "client" guests all boot from that disk and use it for the core system. To perform maintenance on the OS, there are swappable copies of this disk. When applying mantenance, mount the alternate disk, '<span style="font-family: "courier new" , "courier" , monospace;">chroot</span>' into that filesystem, mount any required support, and run '<span style="font-family: "courier new" , "courier" , monospace;">nord-build-csc</span>' for each package of interest. (Tool chain is usually not part of the core OS in NORD, so would be mounted from a separate disk or filesystem.)<br />
<br />
<br />
Packages updated lately include (listed here with versions) ...<br />
<br />
<br />
<ul>
<li>bash-4.4</li>
<li>coreutils-8.27</li>
<li>curl-7.54.1</li>
<li>dash-0.5.9.1</li>
<li>diffutils-3.6</li>
<li>file-5.31</li>
<li>findutils-4.6.0</li>
<li>gawk-4.1.4</li>
<li>gettext-0.19.8</li>
<li>grep-3.1</li>
<li>gzip-1.8</li>
<li>m4-1.4.18</li>
<li>nano-2.8.6</li>
<li>readline-7.0</li>
<li>rsync-3.1.2</li>
<li>sed-4.4</li>
<li>texinfo-6.4</li>
<li>wget-1.19</li>
<li>xz-5.2.3</li>
<li>zlib-1.2.11</li>
</ul>
<br />
<br />
For each, the stub makefile can be found from ... <br />
<br />
<br />
<div style="text-align: center;">
<span style="font-family: "courier new" , "courier" , monospace;">http://www.casita.net/pub/cscratch/</span><i>package</i><span style="font-family: "courier new" , "courier" , monospace;">.mk</span> </div>
<br />
<br />
The syntax of '<span style="font-family: "courier new" , "courier" , monospace;">nord-build-csc</span>' does not include the version. The version is in the stub makefile. <br />
<br />
<br />
After updating core packages, simply unmount the '<span style="font-family: "courier new" , "courier" , monospace;">chroot</span>' environment. Swizzle the alternate disk (now with the latest core packages) into the place of the production disk when the time is right and reboot the client virtual machines. <br />
<br />
<br />
<br />
<b>NORD Updates - supplemental software</b><br />
<br />
<br />
Supplemental packages on NORD are handled primarily via Chicory. (In fact, a major purpose of NORD is to serve as a showcase for Chicory and as a build environment.) Most of the core OS packages in NORD can also be built with Chicory. All of the above updates are reflected in the Chicory build of those packages. For each, see the "wrapper" makefile ... <br />
<br />
<br />
<div style="text-align: center;">
<span style="font-family: "courier new" , "courier" , monospace;">http://www.casita.net/pub/</span><i>package</i><span style="font-family: "courier new" , "courier" , monospace;">/</span><i>package</i><span style="font-family: "courier new" , "courier" , monospace;">-</span><i>version</i><span style="font-family: "courier new" , "courier" , monospace;">.mak</span> </div>
<br />
<br />
Chicory packages updated recently include ...<br />
<br />
<br />
<ul>
<li>bind-9.11.1</li>
<li>db-6.0.20</li>
<li>gcc-4.2.4</li>
<li>gcc-4.8.5</li>
<li>gnucobol-1.1</li>
<li>jansson-2.10</li>
<li>libevent-2.0.22</li>
<li>libevent-2.1.8</li>
<li>libressl-2.5.5</li>
<li>libressl-2.6.0</li>
<li>musl-1.1.16</li>
<li>openssl-0.9.8k</li>
<li>openvpn-2.3.15</li>
<li>python-2.5.2</li>
<li>regina-3.9.1</li>
<li>screen-4.6.1</li>
</ul>
<br />
<br />
This is not an exhaustive list.<br />
<br />
<br />
On NORD, use '<span style="font-family: "courier new" , "courier" , monospace;">nord-build-opt</span>' or '<span style="font-family: "courier new" , "courier" , monospace;">chicory-build</span>' to build Chicory packages. <br />
<br />
<br />
The updated core packages have been reflected in a master spreadsheet for NORD. The supplemental packages have not, but soon will. This spreadsheet, along with other docs, resides in Google space, which is considered sub-optimal since it is proprietary. There is also a Github respository for NORD which includes a growing body of NORD build scripts. The official repository is <span style="font-family: "Courier New",Courier,monospace;">http://www.casita.net/pub/nord</span> and <span style="font-family: "Courier New",Courier,monospace;">http://www.casita.net/nord</span>. (casita.net is available both HTTP and HTTPS and does not require HSTS.) <br />
<br />
<br />
-- R; <><<br />
<br />
<br />
<br />
<br />
<br />Rick Trothhttp://www.blogger.com/profile/03475933977328026233noreply@blogger.com2tag:blogger.com,1999:blog-2506532916329743518.post-57716748598827533042017-07-01T14:26:00.001-07:002017-07-01T14:26:43.987-07:00Identity Based Encryption<div dir="ltr">
<b>Identity Based Encryption</b></div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
We were readying the house for visitors and cleaning-up. In the loft, on a bookshelf, I found a house key. It fit the door to our house, but didn't unlock it. <br />
Where did this key come from? What door does it open? </div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
It occurred to me that some kind of identifier would help. We've got several keys for other houses and they're all tagged. This key happened to not be tagged. </div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
This is one aspect of Identity Based Encryption. With crypto, there are keys. What files, messages, or systems do the keys unlock? The identity is more important than the key itself. </div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
It's not a stretch to say that <i>a key without an identity is worthless.</i> Indeed, a key under your control without an identity that others would recognize may seem more secure. The fact that others cannot know what door (file, message, system) it unlocks may be a kind of security. </div>
<div dir="ltr">
But you know what it's for. </div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
My employer sells "identity based encryption", both asymmetric (IBE) and symmetric (IBSE). It's good stuff. It makes sense. </div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
This unlabeled house key brings the story to life. Keys need identities so their purpose can be known. </div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
-- R; <>< </div>
<div dir="ltr">
</div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
</div>
<div dir="ltr">
</div>
<div dir="ltr">
</div>
<div dir="ltr">
<br /></div>
Rick Trothhttp://www.blogger.com/profile/03475933977328026233noreply@blogger.com1tag:blogger.com,1999:blog-2506532916329743518.post-31128765326639608422017-04-24T05:36:00.002-07:002017-04-24T05:36:53.219-07:00NORD and stali<b>NORD and stali</b><br />
<br />
My friend Skippy sent email suggesting I check out <a href="http://sta.li/" target="_blank">stali</a>. I had seen it before. Good stuff, if perhaps a bit knee-jerk.<br />
<br />
Can't help but think that stali springs from a Linux-centric experience, even though the developers clearly value simplicity. And I'm <i>thrilled</i> to see another team fighting the good fight against bloat.<br />
<br />
How does it relate (if at all) to NORD?<br />
I took the framework of stali goals and wrote a NORD philosophy (section of the <a href="https://docs.google.com/document/d/1QQyCdRe4oHvxFv_23u1DX1rO7EVzhreB6zC3OoCPycM" target="_blank">intro doc</a>):<br />
<br />
<ul>
<li>Follow the Unix philosophy.</li>
<li>Target i386, s390, ppc, and arm hardware.</li>
<li>Separate easily-replaced core system from optional packages.</li>
<li>Follow Linux FHS where it makes sense.</li>
<li>Don’t use SystemD.</li>
<li>Make as much static as is reasonable. (minimize shared library dependencies)</li>
<li>Achieve simplicity and stability. (good rescue or embedded system)</li>
<li>Achieve runs-from-ROM.</li>
<li>Minimize security attack surfaces.</li>
<li>Include a hand selected collection of the standard tools.</li>
<li>Upgrade/install using RSYNC; no package manager needed.</li>
</ul>
<br />
In recent weeks, I needed a clean development system (again!). Had trouble building Squid Proxy on I386. (It built just fine on S390.) So I went about cycling through the core packages (again!), also updated the kernel headers, and tried a re-build of GLIBC. Got stuck. Still stuck. But most things build and re-build just fine.<br />
<br />
I've been reviewing some <a href="http://www.casita.net/chicory" target="_blank">Chicory-built packages</a> to re-do them with static linkage. That will be an ongoing process. <br />
<br />
-- R; <><<br />
<br />
<br />
<br />Rick Trothhttp://www.blogger.com/profile/03475933977328026233noreply@blogger.com0tag:blogger.com,1999:blog-2506532916329743518.post-9504340627794297142017-01-15T13:25:00.000-08:002017-01-18T03:44:57.981-08:00NORD Rationale<b>NORD Rationale</b><br />
<br />
<br />
A few days ago (this is mid January 2017), I added a rationale section to the <a href="http://www.casita.net/pub/nord/doc/nordplan.rtf" target="_blank">NORD Linux intro document</a>. I've given a lot of thought to the "why should anyone bother?" question with respect to NORD. The project has become an obsession, but I find objective reasons to continue using it. It's not just a hobby but a tool for other work.<br />
<br />
Two systems handling web traffic and other services for casita.net are NORD systems. Those are <a href="http://buckeyes.casita.net/about.html" target="_blank">Buckeyes</a> and <a href="http://ltroth1.casita.net/about.html" target="_blank">ltroth1</a>. There are other systems where NORD runs in '<span style="font-family: "courier new" , "courier" , monospace;">chroot</span>' handling some workloads within that jail. So the environment has become significant within this domain.<br />
<br />
<br />
<b>Showcase for Other Projects</b><br />
<br />
<br />
NORD didn't start out as yet another distro. Projects like simply recompiling the Linux kernel added to a collection. The collection grew into a usable system. It reached critical mass and could sustain real work. That seems to be still its primary purpose. It is the stage where other projects perform. <br />
<br />
CSCRATCH is the project which [re]builds the core operating system. It's unique to NORD (though it's not exclusively for Linux). <a href="http://www.casita.net/chicory/" target="_blank">Chicory</a> is more widely effective. Both are just wrappers around the standard recipe.<br />
<div>
<br /></div>
<div>
NORD is my primary platform for hardening activities: reliability, auditability, servicability, as well as penetration defense. There are also growing concerns about trusting trust. (Sure, I'm paranoid. But am I paranoid enough?)<br />
<br />
<br />
<b>Serious about Source</b><br />
<br />
<br />
I've been working with Free and Open Source software for most of my career. At this point, the pay-for and proprietary software that puts food on my table depends inseparably on FOSS. </div>
<div>
<br /></div>
<div>
Ironically, the rise in use of FOSS throughout the industry has not led to a corresponding use of source code at delivery points. The Linux distributors and forward looking software vendors have done such a good job of embracing FOSS and making it drop-in usable that their customers don't need to actually use the source. But it's tech debt. </div>
<div>
<br /></div>
<div>
As long as the providers do the right thing, their customers can proceed with their own business and focus on more important details of that operation. And most vendors/distributors are doing the right thing and will likely continue. But take note, be aware, and get ready. Consider source code as part of your business continuity plan. </div>
<div>
<br /></div>
<div>
NORD can be acquired and used without ever worrying about source or compiling or building. (Compared to "real" distributions it's really rough and I presume the consumer knows traditional Unix.) But NORD is designed to be re-built in a pinch. NORD can assimilate an update or patch faster than any other Linux implementation. (In recent history was Shell Shock which NORD handled no delays: as soon as patches were available, just recompile.) </div>
<div>
<br /></div>
<div>
<br />
<b>Summary in Three Ss</b></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
Rationale for NORD solidifies into simplicity, showcase, and source code. </div>
<div>
<br /></div>
<div>
Now I need to get others interested. A half dozen friends have lent a hand over the course of this saga. A couple of them have been able to make use of the deliverables. We need more participants. (We at least need people to hammer on the build logic, test the results, find bugs and maybe squash them.) </div>
<div>
Wanna help?<br />
<br />
<br />
-- R; <><<br />
<br />
<br />
<br />
<br />
<br /></div>
Rick Trothhttp://www.blogger.com/profile/03475933977328026233noreply@blogger.com0tag:blogger.com,1999:blog-2506532916329743518.post-65556849658153465412016-08-21T12:24:00.000-07:002016-08-22T08:02:04.489-07:00The Death of Veracity<div dir="ltr">
<b>The Death of Veracity</b></div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
Your vocabulary word for today, students, is "<a href="https://en.wikipedia.org/wiki/Honesty">veracity</a>". You may notice that Wikipedia re-directs to the page for "honesty". But this post is about a computer, not about moral integrity. </div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
How we came to name the machine "veracity" is a sentimental saga for another season. The soggy sadness of today is the fallout from Veracity (the machine) having silently succumbed to an as-yet unknown hardware failure. </div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
In recent weeks I've gotten a number of alarms "you need to take backups". We seem to have had a rash of machine failures. Veracity's demise is the second outage this week alone. (The other was "the day the WiFi died" which I'll describe in a separate post.) </div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
This is a disaster <i>recovery</i> story, a D/R tail with a happy ending. </div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
<b>Veracity and Virtualization</b></div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
The good news is that the systems hosted on veracity appear to be intact. One of them, zechariah by name, is our primary IPv6 gateway server. It is up again, after copying of its system disk and root disk to another virtualization host. Whatever failed on Veracity, thankfully the disk was okay. </div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
The other guest, Jeremiah, followed soon after. It acts as our "main" server (email, files, print, DNS, DHCP). But I had gotten lax about backups. The D/R plan for jeremiah was that if it failed we'd switch the IP address for main over from Jeremiah to Nehemiah. While it lived, Nehemiah contained regular backups of Jeremiah's content. We did switch between the two once or twice in those days. </div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
This method of using a similar-but-not-identical system for failover goes back before we had virtual machines on our little network. Where physical systems are involved, the historical plan for D/R is to have another system with the same or better capability standing ready to pick up the load. I was introduced to virtualization in 1982 but pervasive PC-think prevented me from applying tried and true <a href="https://en.wikipedia.org/wiki/Virtualization">V12N</a> methods to personal systems. Bummer. </div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
It began to dawn on me that we don't need a secondary server for a virtual system. All we really need is <u>a copy of that system</u>, a clone. Call it replication. Then when disaster strikes, bring up the clone on a designated alternate hypervisor: no moving around of IP addresses, no quirks from the subtle differences between the recovery system and the real system. A copy of a virtual machine is a perfect substitute because it's <i>not</i> actually a substitute. They're more identical than twins. </div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
<b>Replication and Recovery</b></div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
Zecharian and Jeremiah are in better shape now than they were before the mishap. The host hardware to which they got moved has <a href="https://en.wikipedia.org/wiki/Kernel-based_Virtual_Machine">KVM</a>. Previously they were <a href="https://en.wikipedia.org/wiki/Xen">Xen</a> guests. Not complaining about Xen, but the change forced me to make some adjustments that had been put off, things that needed to be done anyway. They were already configured to share the O/S (another fun rabbit trail, maybe another blog post). They share a common system disk image, now corrected for KVM. (They each have their own root filesystem.) Once the KVM changes were done for one, the other instantly got the same benefit.<br />
<br />
</div>
<div dir="ltr">
I had more success recovering Zechariah and Jeremiah than with this blog post. (Don't get me started about how the Blogger app likes to lose draft updates.) </div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
<b>NORD to the Rescue</b></div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
As it happened, I had a newer kernel for <a href="https://docs.google.com/document/d/1mwIMqmf1f2xd3pTHeCw7vhaodxECZizus2js9gt95H4">NORD</a> than that of the SUSE release Jeremiah and Zechariah run. As it happened, I already had a KVM-bootable system disk. So I copied NORD's kernel to the SUSE system disk and re-stamped the bootstrap. Generally, one simply brings in the kernel and related modules. As long as the kernel and modules are as-new-as and within the same generation the userland parts should have few problems. Works. <br />
<br />
Note: this is a good reason to sym-link <span style="font-family: "courier new" , "courier" , monospace;">/lib/modules</span> to <span style="font-family: "courier new" , "courier" , monospace;">/boot/modules</span> and commit to a replaceable <span style="font-family: "courier new" , "courier" , monospace;">/boot</span> volume It's a feature of NORD but trivial with any Linux distro. </div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
KVM performance sucks relative to that of Xen. Any time you can use para-virtualization (instead of full hardware simulation) you're going to see better performance. Xen was originally para-virt-only and continues to strongly support para-virtualization. But we're using KVM for the sake of manageability. The guests can run with no knowledge of the hypervisor. (We can always switch to para-virt later, selectively per device.) And these guests aren't doing heavy multi-media work. Performance is sufficient. Presence is paramount. </div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
You can see <a href="http://zechariah.casita.net/">Zechariah</a> for yourself. (You'll need IPv6 to hit it.) The web content is presently unchaged, demonsntrating the effect of a replicated virtual machine. An update with part of this story will be forthcoming. Jeremiah's connectivity is more controlled, not generally reachable from outside. </div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
-- R; <><</div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
<br /></div>
Rick Trothhttp://www.blogger.com/profile/03475933977328026233noreply@blogger.com1tag:blogger.com,1999:blog-2506532916329743518.post-42041573229101618392016-07-01T18:04:00.001-07:002016-07-01T18:04:36.516-07:00misdirected redirection<b>misdirected redirection</b><br />
<br />
... or <b>HTTPS everywhere getting us nowhere</b><br />
<br />
<br />
I'm a long time believer in crypto. (Look me up. I'm in the Web-of-Trust.) I've done SSL stack development (in assembler). My day job is helping customers integrate field-level encryption. And I look forward to a safer, more secure, heavily crypto-laden internet.<br />
But I'm concerned about the rush to HTTPS everywhere.<br />
<br />
<br />
Okay, "getting us nowhere" is an exaggeration, but it makes for a decent sound-alike to English speakers.<br />
<br />
<br />
To be specific, there are cases where plain HTTP makes more sense. Sometimes it's just for performance sake. Don't presume that every object must be fetched via SSL/TLS. Some content is actually not a security risk. I could ask "Why would you burden not-at-risk clear traffic with encryption?". But I won't ask. To ask would be rhetorical and that would be ASSuming things that might not be correct.<br />
<br />
<br />
And then there's automation. For better or worse, our world runs on automation, much of which is not encrypted. Not now anyway. <i>Much</i> not now; <i>some</i> not ever. And unencrypted operation is <u>not inherently evil</u>.<br />
<br />
<br />
Rule number one: don't break stuff. <br />
<br />
<br />
Instead of rhetorically asking or assuming, I'll tell you: <b>think</b> about what should and should not be protected with TLS/SSL. Don't blindly bloat our treasured traffic with careless crypto. Think. Be selective about which sessions and services actually needs the extra work. (And it will be extra work, and it won't be your burden alone. Choices you make affect other people, always.)<br />
<br />
<br />
Okay, "don't break stuff" is trumped by security (and by bugs). But hear my point that getting it right is <i>hard work</i>. Blanket solutions aren't solutions, and solving even the most urgent problems by wanton breakage is to follow one problem with another.<br />
<br />
<br />
It's as if someone (make that plural, <i>many</i> someones) asked (rhetorically), "Why would you <b>not</b> encrypt everything?". The question ASSumes that there's no good reason to have cleartext on the net. But there <i>are</i> good reasons. I'll cite only one because I'm tired, presently annoyed, and generally cranky of late. Here's a classic, "Why would you ever need more than 640K?".<br />
<br />
<br />
Important note: That's not how the quote goes and Bill Gates never actually said such a thing. (Someone did say something once to John Sculley about floppies being all Apple would ever need in response to a question about Macintosh networking. Wanna guess who?) Please hear my second point that there is no one-size-fits-all for software, or for hardware, or for clothing.<br />
<br />
My case is the chicken-and-egg situation of trying to build OpenSSL from source. One must download the source before one can build the source. How does that happen? Easy, use '<span style="font-family: "courier new" , "courier" , monospace;">curl</span>' or '<span style="font-family: "courier new" , "courier" , monospace;">wget</span>' or some similar tool, point to openssl.org, get the tarball. Explode the downloaded tarball and follow the standard recipe. But if you don't have SSL working then you can't use HTTPS.<br />
<br />
So far my case doesn't sound like a problem. Here's the problem. Some bright "HTTPS everywhere" aficionado decided that nobody should be using HTTP. When you hit openssl.org via HTTP you (now) get re-directed to HTTPS. If you're downloading OpenSSL for the sake of implementing OpenSSL ... and you don't already have some kind of SSL or TLS ... this is a difficult situation.<br />
<br />
They broke stuff. They broke <i>my</i> stuff. Now it's personal.<br />
<br />
Building systems from source is important. Or I don't know, maybe it's not important. (Seems like it's important to some people, but they're getting hard to find.) I build from source for several reasons, partly because I'm a control freak, partly because I'm a tinker, and partly because I <u>don't trust</u> systems built by other people. Trust ... <u>it's always about trust</u>. (Systems built by other people: I do use them, but kind of like Google. I use Google but I don't trust them. Been saying that for several years now. And look, here I am blogging on a Google property.)<br />
<br />
The automation in question actually has SSL. The problem is that it has an embryonic infrastructure with an empty PKI trust store. This is not to say that it doesn't have a solid trust chain. It just doesn't (at the point of fetching OpenSSL source) have a cache of root certificates for the World Wide Web. So when we hit a site like openssl.org (via HTTPS) the server certificate fails to verify. (Plain HTTP is fine, <i>was</i> fine, until mister "my solution works for everyone" did the re-direct re-design on their site.)<br />
<br />
Gimme back HTTP!<br />
<br />
It's the re-direction that's the problem.<br />
I said HTTP because I meant HTTP. The protocol has 301 and 302. Oy vey, another great feature now fallen victim to abuse by ill-conceived implementation. (There's a long and growing list of those.) The files didn't move. (Would be nice if some people used 301 to replace 404, ya think?)<br />
<br />
This is the second time in recent months that I've run up against someone having disabled a perfectly reasonable function because they knew better than the rest of us. I guess we gotta kill stupidity one bad idea at a time.<br />
<br />
Let's encrypt. Let's encrypt widely. Let's encrypt carefully.<br />
<br />
<br />
-- R; <><<br />
<br />
<br />
<br />Rick Trothhttp://www.blogger.com/profile/03475933977328026233noreply@blogger.com0tag:blogger.com,1999:blog-2506532916329743518.post-84025064729225738742016-06-28T04:57:00.001-07:002016-06-28T04:57:52.288-07:00Off-site Backup<div dir="ltr">
<b>Off-site Backup</b></div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
I've been pressed for time. Scheduling one thing to finish when another is due to start has become essential. On this particular evening, I was hungry, but needed to walk the dog. So I checked the time required for my pizza: 18 to 21 minutes in the conventional oven. Perfect! I could let this cook while exercising the canine. </div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
As I left the house, it occurred to me that I was leaving the hot oven unattended: wife was visiting family, son was on campus, daughter was at work. The risk was small, but it's the kind of risk that we avoid. I was reminded of <a href="https://www.youtube.com/watch?v=lJcnllq3VYY">what happened to Bdale Garbee</a> a year or three ago. </div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
The Garbee family lost everything. Their home was consumed by the same Colorado fire the rest of us heard about on the nightly news. See the video. Bdale's account is enlightening. </div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
For me, this was just a ten minute mental exercise, but here's how it went. All that was most precious was off-premises: family in distributed locations, dog with me. What remained, except for a few heirlooms, was replaceable. But the data? What if we happened to lose the data? Maybe we depend too much on computers to hold our "data". </div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
Until maybe three months ago, I did have off-site backup. Probably seems like overkill for a residential "data center", but I'm a hobbyist. We hobbyists do things for fun that others do only for pay. Dad was more than happy to let me park a surplus desktop-turned-server at his house. A little Linux, shiny new SATA drive, some SixXS to avoid NAT, and a touch of RSYNC. Voi-la! Instant off-site backup. </div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
That was when he <i>had</i> a house. Now he's in an apartment. The facility doesn't provide wired internet; everything is WiFi. And the hack I had rigged to piggy-back that server off of his Windows box began to fail. (First to go was an old 8-port hub in the Rube Goldberg scheme I concocted.) I retrieved the machine several weeks before my lonesome pizza and puppy party. </div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
Surplus hardware is great! You get extended life (from an investment <i>someone</i> made, if not you yourself) and you get low cost service for all kinds of things. Here I had Fedora with LVM and a decent sized platter stack. It was more than just remote storage; it was also a remote point-of-presence. (Helps for those when-not-if times that something funky is happening with the web. And <i>Netflix can just chill</i> because the bandwidth is way to low for regional masking.) </div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
The point of the post: think about a surplus box built to your own specifications as a means to <u>have your own off-site backup</u> or similar service. </div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
But my off-site machine, and its spinning rust, was back home now. Any catastrophe which might wipe out my primary systems would do just as much to my spare. Scary! </div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
When pooch and I got back from our walk, the house was not in flames. There was no smoke nor out-of-control cookery. Instead I was greeted by the aroma of a nearly finished Red Baron Supreme with thin crust. Yesss!! But I am reminded that I need to <i>get serious</i> about the remote box (or maybe two?) and re-deploy <i>real soon now</i>. </div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
-- R; <><</div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
<br /></div>
Rick Trothhttp://www.blogger.com/profile/03475933977328026233noreply@blogger.com0tag:blogger.com,1999:blog-2506532916329743518.post-53799885750250573382015-07-12T20:12:00.000-07:002015-07-12T20:12:08.710-07:00Going Dark<b>Going Dark</b><br />
<br />
<br />
Two scary stories made the GMA headlines Thursday morning. The first breaking news was about the coincidental glitches yesterday (knocking down WSJ, halting NYSE, and grounding Continental/United). People, it really could be just a coincidence . (More on that later.) The second nail biter concerned <a href="http://www.washingtontimes.com/news/2015/jul/7/fbi-encryption-fosters-furtive-terrorism/" target="_blank">FBI director James Comey's warning: crypto cripples canvassing</a>. I often wonder why I take my cues from ABC, and you would be reasonable to ask "why?". But this was a real story; here's why. <br />
<br />
<br />
It's called "going dark" when two parties (good or bad) utilize end-to-end encryption. Directory Comey wants a way around it for cases where one of the parties deserves surveillance. A lot of people in my world are doing a facepalm. This is old news. It's the same thing we've <a href="https://en.wikipedia.org/wiki/Clipper_chip" target="_blank">heard from the gubmint before</a>. Computer security professionals call it "exceptional access" where a police agency (anyone really) can bypass encryption, somewhat like a traditional wiretap. <br />
<br />
<br />
<b>When Crypto is Outlawed</b><br />
<br />
<br />
It's cliche, but you get the idea. <u>When encryption is outlawed, only outlaws will have encryption.</u> This debate is not new. Techies have been fighting in support of cryptography for decades. If you're not a computer geek, you probably did not know. (If you're not a computer geek, you probably don't *want* to know.)<br />
<br />
<br />
Today's chilling warning is frustrating: I kind of like Director Comey, but his testimony before congress <a href="http://news.yahoo.com/fbi-chief-comey-says-strong-encryption-diminishes-agencys-184507873.html" target="_blank">appears to be without proof</a>.<br />
<br />
<br />
<div style="text-align: center;">
"Comey didn't offer any evidence regarding FBI investigations thwarted by strong encryption technology. Still, he said, he felt like researchers and tech companies haven't given the government's request a fair shot before dismissing it. ..." <br />
-- Yahoo news </div>
<br />
<br />
I'd like to introduce Mr. Comey to Lynn Wheeler, Bruce Schneier, Jacob Appelbaum, ... <a href="http://dspace.mit.edu/bitstream/handle/1721.1/97690/MIT-CSAIL-TR-2015-026.pdf" target="_blank">and other computer experts</a>. Yes, James, they <i>have</i> given the government's request <i>a lot</i> of thought, and "dismissed" is the wrong word to describe their response. (I know Wheeler personally. Schneier and Appelbaum have high profile reputations in the crypto world, as do Diffie, Rivest, and other contributors to <a href="http://dspace.mit.edu/bitstream/handle/1721.1/97690/MIT-CSAIL-TR-2015-026.pdf" target="_blank">the MIT paper</a>. These people are the best and brightest on this topic.) <br />
<br />
<br />
<b>When Guns are Outlawed</b><br />
<br />
<br />
There's a similar debate about gun control. I have many friends and family who are in the "right to carry" camp. Guns are dangerous, but lots of things are dangerous. My pro-carry peeps get it that the bad guys <i>will</i> have guns. It's better to arm (and train!) the good guys. <br />
<br />
<br />
Substitute "cryptography" for "guns", or substitute "privacy" there. When guns/crypto/privacy is outlawed only outlaws will have guns/crypto/privacy. The same argument applies (and has yet to be countered) that the bad guys will still have and use guns and crypto even after the law abiding public are disarmed. <br />
<br />
<br />
The difference (for purpose of analogy) between guns and encryption is that guns are physical and cryptography is mathematics. <b>Any back-door means of bypassing crypto is like an invisible hand on (radio control of) the safety on your Glock.</b> Doesn't matter if the manufacturer swears they'll only ever give that control to legit law enforcement, the mechanism itself is a detriment. Experience has proven time and again that exceptional access leads to unthinkable flaws in the resulting products. <br />
<br />
<br />
<br />
<b>Breaking and Entering</b><br />
<br />
<br />
The matter is very personal! I may be damaging my professional reputation by revealing this: I've been hit. It happened a long time ago, and I've learned a lot since then. Someone broke in to one of my computers. Was I surprised, scared? Heck yeah! So I did some forensics and I'm confident about the extent of the breach and have worked hard to keep things secure going forward. <br />
<br />
<br />
Do I want strong encryption? You betcha!!<br />
<br />
But a back door? No way! The front door is enough trouble to guard, thank you.<br />
<br />
<br />
The FBI wants back doors in the apps and systems (phones). By analogy, these apps and smart phones are akin to a house with cheap siding, too many windows, and no Tyvek. They don't even need the barrel of a tank turret to pierce the wall, just kick it. In other words, the infrastructure surrounding two end-to-end encrypted devices is fully exposed and easily watched. <br />
<br />
<br />
<br />
<b>Good Morning America</b><br />
<br />
<br />
It was a GMA headline that jolted me awake. (Coffee had not kicked in yet.) Whether ABC, CBS, NBC, Fox, CNN, ... attention grabbing headlines sell. There's no way "you" are going to get the straight scoop about crypto tech from a major news outlet. It's not in their best interest to give you the full fair-and-balanced story. (Not even Fox, much as they tout that slogan.) If you want to really understand, you may have to do some journalistic gymnastics of your own.<br />
<br />
<br />
Balanced against the consumerism of ABC is <a href="http://www.huffingtonpost.com/victoria-espinel/accepting-director-comeys_b_7752562.html" target="_blank">an article on Huffington Post</a>. Not my favorite rag by a long shot (and the author supports "IP enforcement" raising an eyebrow), but truth is truth and Ms Espinel got this one right: back doors weaken defenses. <br />
<br />
<br />
<b>Good Grief America!</b><br />
<br />
<br />
Being a software guy, with decades of experience in encryption, I knew the GMA story told only the most seductive fraction. Director Comey would do better to mute some of the conversation he wants to start. It's not in our best interest to be arguing details in the news media where terrorists can watch.<br />
<br />
<br />
The fact is, even when a bad guy scrambles the message, he still leaves tracks. Maybe my homey Comey should pal-up with the NSA. Whether they can un-scramble the eggs or not, they're the first people he should ask how-to. The exposed infrastructure (mentioned above) is leaking info all over the place. Google knows it. Apple knows it. The NSA and CIA know it. Chances are the FBI knows it too. <br />
<br />
<br />
Everyone (not just FBI): quit looking for short-cuts.<br />
Going dark does not make the parties invisible, just opaque. <br />
Arm the law abiding majority, including the cops, and get busy with actual police work to stop the criminals.<br />
<br />
<br />
-- R; <><<br />
<br />
<br />
<br />Rick Trothhttp://www.blogger.com/profile/03475933977328026233noreply@blogger.com0tag:blogger.com,1999:blog-2506532916329743518.post-90778037190925448742015-06-01T10:30:00.002-07:002015-06-01T10:46:42.651-07:00Bad Idea #8 - BTRFS Subvols<div dir="ltr">
<b>Bad Idea #8 - BTRFS Subvols</b></div>
<div dir="ltr">
<br></div>
<div dir="ltr">
Got your attention? </div>
<div dir="ltr">
If you like Btrfs then I expect so. </div>
<div dir="ltr">
But take heart: the problem here is not subvolumes. This is not a slam on subvols nor on Btrfs itself. The problem is <u>subvolume <i>abuse</i></u>. Hopefully you'll keep reading and not click away. </div>
<div dir="ltr">
</div>
<div dir="ltr">
Like so many other things, Btrfs subvols are more good than bad, but side effects can have unexpected results, unintended consequences. Here's a quick story of one case of a particularly annoying side effect. </div>
<div dir="ltr">
</div>
<div dir="ltr">
I'm embracing Btrfs for a lot of reasons, so I chose it as the root filesystem type when doing a recent Linux installation. Snapshot capability was also selected by default. I knew about that feature but did not know about some of its consequences. Then when signing on to the shiny new system, I found more than a dozen subvols mounted in support of snapshot. </div>
<div dir="ltr">
</div>
<div dir="ltr">
This broke my RSYNC plan. </div>
<div dir="ltr">
</div>
<div dir="ltr">
My usual habit is to install the new system in a virtual machine, then copy ... er, uh ... take a snapshot (ha!). The copy gets put into a dedicated logical volume. I then copy the new kernel and initrd to a common boot partition used by all systems. This works really well: I get a variety of distros and releases and yet can '<span style="font-family: "Courier New",Courier,monospace;">zypper</span>' or '<span style="font-family: "Courier New",Courier,monospace;">yum</span>' as needed within each. </div>
<div dir="ltr">
</div>
<div dir="ltr">
This time, there were Btrfs subvolumes. The subvols showed as individual mount points, so the "-x" flag did the right thing by not descending across mounts. We wouldn't want to snag <span style="font-family: "Courier New",Courier,monospace;">/proc</span> or <span style="font-family: "Courier New",Courier,monospace;">/sys</span> when snapping that copy of the new op sys, so we really need that "-x". But we do want the normal files, all of them. We don't want to have to construct a myriad '<span style="font-family: "Courier New",Courier,monospace;">rsync</span>' invocations to make it work. </div>
<div dir="ltr">
</div>
<div dir="ltr">
The system I installed, not knowing to de-select snapshots, wound up with 14 subvols. It looks like a case of some developer trying to do with subvols what we all used to do with LVM. (Scaled up just a bit since that number of "volumes" is beyond what anyone bothered with before we had subvols.) Now ... the layering violation (subvols versus LVM) is a whole nutha topic. But you can see from this count: 15 invocations of '<span style="font-family: "Courier New",Courier,monospace;">rsync</span>' versus just one.<br>
<br>
Perhaps just a nuisance?<br>
Fairly, though, a nuisance with questionable requirement. </div>
<div dir="ltr">
</div>
<div dir="ltr">
</div>
<div dir="ltr">
These "bad idea" rants sound horribly whiny. There <b>is</b> an upside. Reader, please understand, it's really about KISS. In this particular complaint, subvolume abuse goes against the grain of simpler filesystem handling. </div>
<div dir="ltr">
</div>
<div dir="ltr">
Where has simplicity gone? </div>
<div dir="ltr">
We in the Linux and open source world love to pick on Windows, yet we're now <u>creating the same complicated crap</u>, monolithic mechanisms with unforeseen flaws. </div>
<div dir="ltr">
</div>
<div dir="ltr">
-- R; <><</div>
<div dir="ltr">
<br></div>
<div dir="ltr">
<br></div>
<div dir="ltr">
<br></div>
Rick Trothhttp://www.blogger.com/profile/03475933977328026233noreply@blogger.com0tag:blogger.com,1999:blog-2506532916329743518.post-1227441319612653942015-04-24T10:34:00.000-07:002015-04-24T10:40:12.113-07:00KVM Across the Board<div dir="ltr">
<b>KVM Across the Board</b></div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
Let's talk about hardware.<br />
Let's talk about physical hardware versus emulated hardware.<br />
Let's talk about virtual hardware.</div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
<b>Emulation </b></div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
Emulation is where you run a program to simulate a machine. A popular PC emulator is BOCHS. You can use BOCHS to boot any operating system which would run on I386 hardware. BOCHS can run on any hardware, I386 or other. BOCHS can run on z Series (IBM mainframe). BOCHS can run on p Series (IBM RS/6K). BOCHS can run on ARM (your cell phone).<br />
<br />
There are many emulators for many architectures.<br />
Mainframe aficionados commonly use Hercules to emulate a mainframe where BOCHS emulates a PC. </div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
There is a steep performance penalty in emulation. Each emulated CPU instruction requires dozens or hundreds of physical CPU instructions. Processors this century are so fast that we sometimes don't notice. (The pain of this performance penalty varies widely for varying workloads.) </div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
<b>Virtualization</b></div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
Virtualization is better. </div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
Virtualization is hard to implement but easy to understand. A "guest" runs on the underlying physical machine until there is an exception (privileged operation, I/O, a page fault, or similar). Emulation does come into play (because not all devices and hardware facilities are virtualized). But virtualization is different from emulation in a big way. With virtualization, the guest runs on the bare metal most of the time, not interpreted by a program. Clearly more efficient. </div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
Something which hosts guest operating systems via virtualization is called a hypervisor. The premier hypervisor is z/VM. z/VM only runs on IBM z Series hardware.</div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
<b>Let's talk about KVM</b></div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
Since VMware made its debut in the late 1990s, virtualization has been more than just a mainframe phenomenon. Until that time, the only serious hypervisor was IBM's VM (variously VM/ESA, VM/XA, VM/SP, and other monickers, now z/VM). Before VMware, the best we got (in quantity) apart from z/VM was emulation. But now there are many. </div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
We cannot know (unless we measure each, which is time consuming) how much true virtualzation each hypervisor really does. There is the concept of paravirtualization, which is beyond the scope of one little blogger post. Let's talk about that L8R. </div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
An incomplete list (just for discussion) of hypervisors includes z/VM, VMware, Xen, KVM, and products from Sun/Oracle, Microsoft, Apple, and other vendors.</div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
But let's talk about KVM.</div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
KVM started life in the PC world, specifically the Linux PC arena. We had Xen, but it was paravirt at best. Advances in chip logic from AMD and INTeL allowed for more effective virtualization.So Qumranet developed KVM and managed to get it accepted into the Linux kernel. Thus it is the <i>kernel</i> virtual machine facility. Nice! </div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
I got wind that KVM was being made to work on z Systems. (Being a late bloomer, the info was after-the-fact.) I also found out that KVM had been ported to p Systems. (That's the PowerPC chip.) </div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
Now ... this is a really cool development: the same hypervisor across multiple hardware architectures. I have imagined this and longed for it for two decades! Wow. Can't do that with VMware. Can't do that with Oracle, Apple, Microsoft. Can't even do that with IBM z/VM. </div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
<b>Let's talk about IBM</b></div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
This little post is in response to a thread on LinkedIn. Specifically, I'm trying to answer Jon Butler's question if KVM is enabled for the mainframe hardware. <u>The answer is yes.</u></div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
The mainframe is amazing and has lots of bells and whistles. At the center is the same basic idea of a general purpose processor (variously called a CP or GP, with a Linux-specific spin called IFL, loosely "S390"). z/VM runs on it. KVM runs on it. </div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
<u>KVM runs on: I386 (and X86_64), PPC, S390, and (soon) ARM</u></div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
But then there's z/VM. </div>
<div dir="ltr">
IBM has this incredible hypervisor. I don't have time to describe the orders of magnitude difference between z/VM and the others.</div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
<u>z/VM runs on: S390</u></div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
There is ample historical evidence that IBM has wanted to kill z/VM for a long long time. Maybe it threatens their cash cow (z/OS, nee MVS). </div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
Many respected colleagues have expressed frustration with IBM in recent months over positioning of their shiny new z13 machine. A z13 can run hundreds, nay thousands, of Linux guests without breaking a sweat. It can over-commit memory and still deliver flawless service with sub-second response time. It can do all this by the magic of z/VM.</div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
So what's wrong?</div>
<div dir="ltr">
What's wrong is that IBM marketing hints the amazing z13 does all this with the help of KVM. That's simply not true. It's like the marketing guys don't know the difference between "z/" and "K". </div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
z/VM is not all that expensive (to you). Might be expensive to IBM.<br />
z/VM is, however, very different from KVM. No doubt some customers with pre-disposed warm fuzzies for KVM will embrace that and expect the results touted for z/VM. Not gonna happen.</div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
<b>What Shall We Do?</b></div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
I love z/VM. There's nothing like it. It takes full advantage of z hardware and then adds its own incredible features.</div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
I love KVM. It breaks out across hardware architectures. But it's a baby, just starting to crawl. </div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
My fear is that IBM will finally lop-off their VM product (like some say they've wanted to do for decades) and leave us with only KVM. </div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
So what shall we do?</div>
<div dir="ltr">
First, <u>be clear about the story</u>. If an IBMer rattles on about z13 and only includes KVM, raise your hand. Call them on the carpet about it. Second, if they do proceed to ditch z/VM (which experienced engineers believe would be a major mistake), then as customers demand that they <u>port all the features</u> of z/VM to KVM: the monitor stream, the performance controls, DCSS, IUCV, the unquantifyable* features of CP/CMS.</div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
*Not sure "unquantifyable" is a word, but you get the idea. </div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
-- R; <><</div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
<br /></div>
Rick Trothhttp://www.blogger.com/profile/03475933977328026233noreply@blogger.com0tag:blogger.com,1999:blog-2506532916329743518.post-62519125105689409672015-04-22T10:18:00.000-07:002015-04-22T10:18:17.657-07:00Bad Idea #75 - Mandatory Indentation<div dir="ltr">
<b>Bad Idea #75 - Mandatory Indentation</b></div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
I've done it! <br />
I've finally learned <a href="http://en.wikipedia.org/wiki/Python_%28programming_language%29" target="_blank">Python</a>. <br />
Could not put it off any longer and what a cool language! </div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
Found out about the origin of the name. Guessing <a href="http://en.wikipedia.org/wiki/Guido_van_Rossum" target="_blank">Guido</a> wanted the Holy Grail of languages. I'm also a fan of the comedy troupe, and MP/HG is one of my favorite films, so you might expect me to love the language just for that. But no. Had to continue with my bad attitude. </div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
Anyone notice that the Dutch are taking over the computing world? And in case GvR ever runs across this, I <i>have</i> been to Haarlem. Beautiful town. </div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
</div>
<div dir="ltr">
I didn't like Python. <br />
It's an aesthetics thing. <br />
(I dislike <a href="http://en.wikipedia.org/wiki/Perl" target="_blank">Perl</a> for the same reason, resulting in similar self injury.) </div>
<div dir="ltr">
</div>
<br />
<div dir="ltr">
<b>Thou Shalt Bathe</b></div>
<div dir="ltr">
</div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
Unlike my friend JEP, I have no allergy to curly braces, so for me there's no <i>need</i> for indentation (as far as making the compiler/interpreter happy). But I <i>do</i> indent. Always have. C'mon ... that's the most basic coding hygiene. (I bathe too.) </div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
What offends me about Python is that indentation is part of the language, that indentation is mandatory. (Somewhere in the history of '<span style="font-family: "Courier New",Courier,monospace;">make</span>' is a similar soggy saga w/r/t indentation. I will cite another time.) </div>
<div dir="ltr">
</div>
<div dir="ltr">
</div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
Look ... I indent as a matter of course, so when I refrain from indenting there's a reason for it. But with Python I don't have that liberty. The language forces me to indent like a parent telling their kid to brush their teeth. </div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
</div>
<div dir="ltr">
<b>A Pearl of a Language</b></div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
Now that I've dived in, this water's fine. Really. </div>
<div dir="ltr">
Writing new code is easy as shell. Yet it is much better structured (than shell) so the results are less of a hack and more like a Mack (truck). It has a strong collection of libraries, including a spiffy OS interface, and supports "the right" set of data types. </div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
Took less than a day to port one of my Tcl scripts to Python. Easy! And fun! </div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
Yet, like any gem, it does have flaws. </div>
<div dir="ltr">
Maybe "flaws" is too strong of a word. Call them "qualities". </div>
<div dir="ltr">
I've been so apologetic about <a href="http://en.wikipedia.org/wiki/Rexx" target="_blank">REXX</a> over the years. Python has similar quirks. (And <a href="http://en.wikipedia.org/wiki/Mike_Cowlishaw" target="_blank">MFC</a> gets a pass on quirks because REXX was created in a vacuum before the glut of scripting languages we now carry.) Some aspects of Python take a bit of getting used to ... in a way that reminds me of REXX. </div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
I already lost one job offer due to lack of Perl skills. </div>
<div dir="ltr">
Hopefully I won't lose any others from lack of Python prowess. </div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
Now looking forward to more Python. </div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
</div>
<div dir="ltr">
</div>
<div dir="ltr">
-- R; <><</div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
</div>
<div dir="ltr">
</div>
<div dir="ltr">
</div>
<div dir="ltr">
</div>
<div dir="ltr">
</div>
<div dir="ltr">
<br /></div>
Rick Trothhttp://www.blogger.com/profile/03475933977328026233noreply@blogger.com0tag:blogger.com,1999:blog-2506532916329743518.post-60191960530038265542015-02-06T16:21:00.002-08:002015-02-06T16:21:31.053-08:00Kitchen Code<div dir="ltr">
<b>Kitchen Code - how to *not* cook software</b></div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
Ah, the joy of writing a new program from scratch. Well, mostly from scratch. We use bits and pieces of other programs and mix it all together, season it just right. Poof ... perfection. That's what programming is all about, right?</div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
Hardly.</div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
Real programming means you're going to come back to a program later. Maybe you need to add a feature. Maybe you need to squish a bug. And maybe it's not as easy as the first time in that stew. This after-the-first-release kind of programming is not as exhilarating as "green field" work.</div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
<b>Other hands in the Soup</b></div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
As if re-working your own code wasn't bad enough, turns out you sometimes have to work on someone else's creation. This can actually be a good thing. This can actually be fun, believe it or not. (You've surely heard about the wonder of Open Source Software. Lots o team work there.)</div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
Coding as part of a team takes discipline. It takes more time at first. (But often takes less time in the fix-up and fancy-up phase.) It's more rewarding.</div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
Working on multi-author code is better ... as long as all authors agree on standards. There are two dangers, problems, with team code that I call "Kitchen Code". Oh how I wish we could all be free from them.</div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
<b>A pot of Spaghetti</b></div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
Spaghetti Code is the perfect term for a program which jumps around. If you've been into programming for any decent length of time, you have surely heard the phrase. You have probably even seen some spaghetti code. You may have even written some. You'll be forgiven. Just don't do it again!</div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
I love lower level languages. Maybe that's why I love C. Some people call C a "glorified assembler". To me that'd be a compliment. But even C provides strong entry/return fencing. Get into real assembly language and there's real temptation.</div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
Assembler provides the most efficient coding possible. It's also tedious. I don't know if it's the temptation to optimize or that coders tire of the tedium, but it's really common to ... heck ... just branch already! So efficient. So easy. But don't.</div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
No! You must be strong padawan!</div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
<b>Dry-Erase board covered with Post-It Notes*</b></div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
Noodle cooks have another tendency: disorganized data.</div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
On the other wall from the stove with that spaghetti pot is the white board. There are sticky notes on it. The notes have little bits of information someone needs.</div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
Notes are great. Scratchpads are awesome. But be sure to boil off the excess liquid and get something solid. By "solid" I don't mean one goopy blob of common storage. Organize it. Scope it. Don't expose the whole beefy lot of bits and bytes to every routine in your casserole.</div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
Large structs with global exposure are like rum in the skillet. Someone gets burned.</div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
But *do* use *real* Post-It notes (er, uh ... comments) to explain what you were thinking. They're like margin notes in your recipe book. Do it. </div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
So remember: when it comes to software, avoid spaghetti and contain the data.</div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
-- R; <><</div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
</div>
<div dir="ltr">
</div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
*Post-It is a registered trade mark of 3M. But we like 3M and we trust they'll appreciate the free advertising and not give us crap for using their TM here.</div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
<br /></div>
Rick Trothhttp://www.blogger.com/profile/03475933977328026233noreply@blogger.com0tag:blogger.com,1999:blog-2506532916329743518.post-14881008742211947072014-11-06T07:13:00.004-08:002014-11-06T07:13:39.966-08:00FuzixOS: Because Small Is Beautiful<br />
<b>FuzixOS: Because Small Is Beautiful</b><br />
<br />
Linux kernel developer Alan Cox has come up with an excellent new toy. In recent months, he has reported working with several older hardware platforms ... just for fun. And now, he has published <a href="https://github.com/EtchedPixels/FUZIX" target="_blank">FuzixOS on Github</a>. <br />
<br />
In the open source world, I've been noticing a loss of simplicity lately. Fuzix is one alternative to the bloat. It's a collection of features from UZI and various Unix for 8-bit systems. Yes, in terms of anti-bloat, this is extreme. But it's cool! <br />
<br />
Can't help but think about <a href="http://docs.google.com/View?id=dc6n97p5_21d4qng3c4" target="_blank">NORD</a> in this context. The prime <a href="https://docs.google.com/document/d/1mwIMqmf1f2xd3pTHeCw7vhaodxECZizus2js9gt95H4" target="_blank">motivator for NORD is the need for simplicity</a>. If Fuzix gets a little more robust, maybe some of the FOSS packages in NORD can be tried on Fuzix. ("Portable Apps" will require sym-links, but the CSCRATCH parts will likely work straight away.) It might even be possible to use the Fuzix kernel for a "real" NORD build.<br />
<br />
<br />
-- R; <><<br />
<br />
<br />
<br />Rick Trothhttp://www.blogger.com/profile/03475933977328026233noreply@blogger.com0tag:blogger.com,1999:blog-2506532916329743518.post-29768409732906904062014-09-22T10:59:00.001-07:002014-09-22T10:59:52.071-07:00Impending Internet<b>Impending Internet</b><br />
<br />
I heard a respected software engineer say something like, "I saw the advent of the web, and that turned out to be right. I saw Linux on virtualization, and that turned out to be right.". Personally I felt a little vindicated because I had seen the same things. But I was frustrated by his comment because this guy also <i>does not see</i> some impending changes of our landscape. <br />
<br />
Impending ... as if something is yet to be.<br />
Case in point, IPv6. I've been pitching IPv6 for the platforms I support for several years. But a number of my peers <u>just don't like it</u>. They don't want to "do" IPv6 until they <i>have to</i>. I gotta think that the procrastination is from ignorance: I suspect that they <i>think</i> IPv6 is difficult. It's not. <br />
<br />
IPv6 is not impending. It's now. "It's <i>soooo</i> yesterday!" <br />
<br />
As for me, I hate being forced into something, even if it's something that I like, something cool or fun. I got into IPv6 with help from the Orc almost four years ago. Had been trying on my own to crack into IPv6 for the better part of a decade. In the end, I got by with a little help from my friend. <br />
<br />
Then today's news, not specifically about IPv6, but tangentially related. I heard about a <a href="http://www.netresec.com/?page=Blog&month=2014-09&post=Analysis-of-Chinese-MITM-on-Google" target="_blank">Chinese government hack against SSL</a>. The targeted network is their CERNET education and research network, which it turns out <u>is entirely IPv6</u> (<a href="http://en.wikipedia.org/wiki/China_Next_Generation_Internet" target="_blank">CERNET2</a>, at least). Here's my chance to say it: <u>told ya</u>. The Asian tech sector is way ahead of Europe and the Americas on this front.<br />
<br />
Most of the major shops have a complete IPv6 rollout underway, many nearly complete. So ... there's plenty of examples of getting this done. C'mon, try with a little help from your friends.<br />
<br />
<br />
<br />
<br />
-- R; <><<br />
<br />
<br />
<br />Rick Trothhttp://www.blogger.com/profile/03475933977328026233noreply@blogger.com0tag:blogger.com,1999:blog-2506532916329743518.post-59488122596388851382014-07-13T16:05:00.002-07:002015-01-17T11:30:32.302-08:00Proposing Powerful Portable Pipelines<b>Proposing Powerful Portable Pipelines</b><br>
<br>
IBM calls the package CMS/TSO Pipelines. A certain group of us affectionately call it "Hartmann Pipes". It's standard equipment on VM/CMS, but a lot of people upgrade to the author's version. On TSO (part of z/OS), it's optional. Those who learn even the simplest of its capabilities tend to get really hooked on it.<br>
<br>
When you're excited about something, you talk about it. But discussing Hartmann Pipelines with the uninitiated is difficult. Most people have seen command line pipes (Unix, Linux, even Windows) so they wonder "what's the big deal?". How is this implementation different from other pipelining schemes? Aye, there's the rub. I'll get into that detail later. And some readers will need that rationale before any of this makes sense. First ... the proposal.<br>
<br>
There are two audiences. There are those who don't know CMS Pipelines. I gotta sell you on this wonderful thing. There are those who do know CMS Pipelines. I gotta sell you on this particular implementation. It includes a compromise; I gotta sell you on the compromise. Maybe from the two groups there will be enough volunteers to achieve critical mass.<br>
<br>
<b>The Proposal</b> <br>
<br>
There have been several ports of pipelines. Prior efforts require one or more substantial support environments. The idea here is to use pure C. Interaction between the stages is summarily handled by these functions ...<br>
<br>
<ul>
<li><span style="font-family: "Courier New",Courier,monospace;">output(p,d,l) /* blocks until record is consumed */</span></li>
<li><span style="font-family: "Courier New",Courier,monospace;">peekto(p,b,l) /* examines a record without consuming */</span></li>
<li><span style="font-family: "Courier New",Courier,monospace;">readto(p,b,l) /* consumes a record (and gets contents) */</span></li>
</ul>
<br>
<span style="font-family: "Courier New",Courier,monospace;">output()</span> is used by a producer stage. The producer is blocked until the record is consumed. <span style="font-family: "Courier New",Courier,monospace;">p</span> is a pipe struct pointer, referencing a connection, in this case an output connector. <span style="font-family: "Courier New",Courier,monospace;">d</span> points to the data, but it's not necessarily a NULL terminated character string. There are no sacred characters. <span style="font-family: "Courier New",Courier,monospace;">l</span> says how long the record is.<br>
<br>
<span style="font-family: "Courier New",Courier,monospace;">peekto()</span> and <span style="font-family: "Courier New",Courier,monospace;">readto()</span> are used by a consumer stage. They're identical except that <span style="font-family: "Courier New",Courier,monospace;">peekto()</span> leaves the record pending, and leaves the producer stage blocked. <span style="font-family: "Courier New",Courier,monospace;">p</span> is a similar pipe struct pointer to that used for <span style="font-family: "Courier New",Courier,monospace;">output()</span>, except that it references an input connector. <span style="font-family: "Courier New",Courier,monospace;">b</span> points to a buffer and <span style="font-family: "Courier New",Courier,monospace;">l</span> indicates its size.<br>
<br>
These are easily implemented on any POSIX system using a pair of traditional pipes. We'll have one Unix style pipe pointed "downstream" sending data and stats, and another pointed "upstream" sending control signals. Some of the control signals are ...<div><div>
<br>
<ul>
<li>tell me about the record</li>
<li>give me the data, used by both <span style="font-family: "Courier New",Courier,monospace;">peekto()</span> and <span style="font-family: "Courier New",Courier,monospace;">readto()</span></li>
<li>unblock, "got it", producer can move along, used by <span style="font-family: "Courier New",Courier,monospace;">readto()</span></li>
</ul>
<br>
The producer accepts these controls via the upstream pipe and sends meta data or data on via downstream pipe. The producer blocks until it gets a "got it" signal from the consumer. The consumer blocks until the producer has a record to send.<br>
<br>
The "connector" struct starts with two file descriptors ...<br>
<br>
<span style="font-family: "Courier New",Courier,monospace;">struct PIPECONN {<br> int ctrl; /* control from consumer to producer */<br> int data; /* data from producer to consumer */</span><br>
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-family: "Courier New",Courier,monospace;"> ... other elements ...</span></span><br>
<span style="font-family: "Courier New",Courier,monospace;"> }</span><br>
<br>
The <span style="font-family: "Courier New",Courier,monospace;">data</span> pipe alternately carries metadata based on signals on the <span style="font-family: "Courier New",Courier,monospace;">ctrl</span> pipe. <br>
<br>
<br>
<br>
<br>
<br>
<b>The Compromise</b><br>
<br>
The above operations are under control of the host operating system. Some time back, I mentioned this idea to the Piper (Hartmann himself). His take on it was not clear. CMS Pipelines has its own dispatcher. (CMS originally did not have built-in multi-tasking.) I've mentioned the idea to several serious plumbers. They unanimously reject it, citing Hartmann's dispatcher as an essential requirement. But is it?<br>
<br>
About the time I first published this article, David Craig shared his excursion into Plan 9 (that's a whole nutha journal entry) including discussion about Plan 9 security. I found this ...<br>
<br>
<div style="text-align: center;">
"... weak but easy-to-use security can be more effective <br>than strong but difficult-to-use security if it is more likely to be used."</div>
<br>
The point goes for more than just security. A weak (or not well tuned) but easy-to-use dispatcher can be more effective if it leads to a simpler implementation. So that's to the Pipes knowing audience. Let each stage be a unique process.<br>
<br>
Now a word to the uninitiated.<br>
<br>
<br>
<b>A Record by Any Other Name</b><br>
<br>
Hartmann Pipes convey records. "Eww ... such a mainframe concept." Well, yes, in fact. John Hartmann invented Pipelines on a mainframe and CMS/TSO Pipes runs on two mainframe operating systems.<br>
<br>
Why do we fight over these things? If it were called a packet or a message or perhaps a block then you wouldn't have a problem with it. I won't enumerate the reasons for using quantized data (records) leaving examples such as MQ and TCP as ample justification.<br>
<br>
The value of the Hartmann implementation includes ...<br>
<br>
<ul>
<li>record structure (if any) is retained</li>
<li>"delaying the record" can be tolerated or avoided, as needed</li>
<li>an entire record can be examined before it is consumed, which allows for in-flight changes to the pipeline based on content</li>
<li>a stage can be added, to input or output, between two connected stages</li>
<li>multiple streams can run concurrently, and fan-in or fan-out</li>
<li>output can be looped back to input</li>
</ul>
<br>
<br>
We could really use this on systems other than only CMS and TSO.<br>
<br>
<br>
-- R: <><<br>
<br><br>Originally published 2014-July-13. <div><br></div><div><br></div><div><br>
<br>
<br></div></div></div>Rick Trothhttp://www.blogger.com/profile/03475933977328026233noreply@blogger.com0tag:blogger.com,1999:blog-2506532916329743518.post-11150339565354652432014-05-27T11:27:00.002-07:002014-05-27T11:27:17.134-07:00Data Sharing versus Data Security<b><span style="font-size: small;">Data Sharing versus Data Security</span></b><br />
<br />
Two of my favorite topics that seem to contradict each other are the sharing of data and the securing of it (especially crypto). I'll be pitching both at the upcoming <a href="http://www.vmworkshop.org/" target="_blank">VM and Linux Workshop</a> next month at <a href="http://www.ncat.edu/" target="_blank">NC A&T</a> in Greensboro. What's really amazing at this time is how hot both topics are and yet how poorly understood they are.<br />
<br />
The VM and Linux Workshop is an highly technical and surprisingly inexpensive conference. This year there will be the added value of <a href="http://velocitysoftware.com/seminar/workshop.html" target="_blank">Velocity Software's performance seminar</a> two days prior to the workshop (at the same location).<br />
<br />
<b>Data Sharing</b> <br />
<br />
The glut of mobile computing and popularity "cloud computing" illuminates the need for data sharing ... even amplifies it. For just one example, think about your list-o-contacts on your smart phone. Handy, eh? But not so handy if you have to go into contortions to synchronize the info with other parts of your digital life. Gotta have data synch, and that's data sharing. <br />
<br />
Full disclosure: the presentation will specifically cover <i>filesystem</i> sharing, though we'll touch on data synch and other such content manglement. Networked filesystems were more common until recently. In a context where applications use <i>files</i>, network sharing of filesystems makes a lot of sense. But even where apps use methods other than files for their working data, they need files for basic operation. And here's the kicker: most of the operating system needs files. So ... sharing of filesystems makes a lot of sense in more contexts than you might think. <br />
<br />
Sharing filesystems as block devices or memory is cooler: It is efficient, effective, even elegant. Sharing virtual disk or virtual ROM is obvious to some but still not practiced as widely as it should be. And sharing disk images works in SAN land as well as in the virtual world. "It's everywhere you want to be." <br />
<br />
<b>Data Security</b><br />
<br />
We're talking crypto, but not <i>just</i> crypto. (Controlling access is crucial.)<br />
<br />
I mentioned smart phone and contacts. Data sec balances data sharing so that your contacts remain yours. <br />
<br />
The presentation will cover a mere handful of the data access issues. (You could have a week long seminar discussing all related topics!) For the focus topic of SSL, it's vital to protect the server's secret key.I'll get into how that works, why its important, and related tools (PGP/GPG and SSH and others). Ciphers and hashes and keys ... oh my! <br />
<br />
Truth be told, PKI is a mess, but it gives us a framework, a reference. The heavy lifting is done by SSL (and TLS) which is really slick, well defined, and robust. But the underlying cryptography is a moving target. As soon as you learn one cipher or hash, you find it has been defeated. So keeping things plug-compatible is essential. <br />
<br />
Both of these topics border on obsession for me. They're so geeky cool. More people should use them. Even where they're not applicable, they should be considered ... part of your "doctor's bag" of remedies and treatments. I'm excited to be talking at the Workshop again. <br />
<br />
-- R; <><<br />
<br />
<br />
<br />Rick Trothhttp://www.blogger.com/profile/03475933977328026233noreply@blogger.com0tag:blogger.com,1999:blog-2506532916329743518.post-9686896227326805352014-05-07T05:08:00.002-07:002014-05-07T05:08:48.879-07:00Sir Santa Seen in Super Set Script<div style="text-align: center;">
Sir Santa Seen in Super Set Script</div>
<br />
<div style="text-align: center;">
or</div>
<br />
<div style="text-align: center;">
Fun with Unicode</div>
<br />
<br />
We suspect this is what Sir Santa really looks like. He even has a <a href="http://www.fileformat.info/info/unicode/char/1f385/index.htm" target="_blank">Unicode</a> point of his own! <br />
<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://1.bp.blogspot.com/-JSoM4hAZw4A/UNfpwrc8ryI/AAAAAAAAAo4/bqcmtVbePos/s400-no/AndroidEmoji-1F385.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://1.bp.blogspot.com/-JSoM4hAZw4A/UNfpwrc8ryI/AAAAAAAAAo4/bqcmtVbePos/s400-no/AndroidEmoji-1F385.png" height="320" width="320" /></a></div>
<br />
<br />
But he's not much of a fan of <a href="https://code.google.com/p/android/issues/detail?id=41827" target="_blank">Android</a>, perhaps since they have "issues".<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />Rick Trothhttp://www.blogger.com/profile/03475933977328026233noreply@blogger.com0tag:blogger.com,1999:blog-2506532916329743518.post-51240144886401140442014-03-05T15:09:00.000-08:002014-03-05T15:15:32.561-08:00Dave Tuttle's Memoirs<b>Dave Tuttle's Memoirs</b><br />
<br />
<br />
I suppose some people think they're in it just for the money or just for the tech or just for the business. Maybe. But human nature consistently shows us that we need something more. Ultimately, it's that "image of God" creativity, even for those who think we're not terribly creative (and those who know we're not terribly god-like or even godly).<br />
<br />
Why do you do what you do?<br />
In software, especially in <a href="http://en.wikipedia.org/wiki/FLOSS" target="_blank">FOSS</a>, why do you code the code you code?<br />
<br />
<b>Intrigue</b><br />
<br />
I attended a Git workshop this past Monday evening. (Yeah, yeah ... it's about time. Hey! It's not like I don't use code manglement, CVS, SVN, etc.) The class was taught by Aidan Feldman of github. Great stuff! Excellent quick workshop; got us all actually using Git on github. Anyway, he mentioned that Linus created Git. I asked if the legend was true ... that Linus did it over a weekend. The <a href="http://en.wikipedia.org/wiki/Git_%28software%29" target="_blank">bastion of all human knowledge</a> says Git was done in four days and fully operational in a matter of weeks. Ahhh... that's the good stuff.<br />
<br />
<br />
Before Linus, there were other heroes: Vint and Vixie, Gettys and Garbee and Grace, Barber and Farber, John and John and Jonn (Eckert and Mauchly -vs- von Neumann, rivals), Brooks and Bartik, Kay and Cray, Englebart. I'm grasping at straws and leaving out dozens ... hundreds. But you get the idea.<br />
<br />
Intrigue inspires and FOSS fosters freedom.<br />
<br />
<b>VM/CMS</b><br />
<br />
In my day job, I work with CMS. No, not that <a href="http://en.wikipedia.org/wiki/Content_management_system" target="_blank">CMS</a>, <a href="http://en.wikipedia.org/wiki/Conversational_Monitor_System" target="_blank">this CMS</a>. It's IBM's "Conversational Monitor System" (fka "Cambridge Monitor System" to show some history). CMS is a single user, single-tasking operating system hosted by the oldest viable hypervisor: VM/CP the "Control Program".<br />
<br />
Hey ... hypervisor ... you know what that is! It's that thing Rosenblum created. Yeah.<br />
<br />
Not exactly. Didn't mean to drip into sarcasm. If I haven't lost the audience, then what follows may be more inspiration than irritation.<br />
<br />
<a href="http://en.wikipedia.org/wiki/VM_%28operating_system%29" target="_blank">VM/CP</a> is arguably the best hypervisor, and CMS (in spite of its warts) has been able to focus on a handful of incredible things, including CMS UPDATE. (We did mention code management, you recall.) I love this stuff!<br />
<br />
Virtualization rocks! With CP under it, CMS is truly useful. XEDIT, REXX, Pipelines, a CISC processor and a full-featured assembler (plus C and other languages), that multi-level source code update tool, even OpenVM (yes, there's a shell, and yes, that part is multi-tasking).<br />
<br />
<b>History of VM</b><br />
<br />
In the glory days of VM, <a href="http://www.leeandmelindavarian.com/Melinda/" target="_blank">Melinda Varian</a> put together the often cited and repeatedly praised "VM and the VM Community: Past, Present, and Future". Her work ranks up there with Tracy Kidder's "The Soul of a New Machine", though less well known (and without the assist of a publisher).<br />
<br />
I was already a fan of VM when I saw Melinda present at a SHARE conference. Hearing the history, the stories of how the system came to be, it all gave me chills. This is the system IBM tried to kill. Over and over.<br />
<br />
The ultimate insult (to IBM) happened when Linux was ported to the mainframe. Linux works great on any platform, but is a teriffic candidate for virtualization, especially on high-redundancy hardware (like mainframes). There were two ports, by the way, one by IBM and another by Linas. No, not that <a href="http://en.wikipedia.org/wiki/Linus_Torvalds" target="_blank">Linus</a>, and not <a href="http://en.wikipedia.org/wiki/Ben_Linus" target="_blank">that one</a> either, <a href="http://www.linas.org/" target="_blank">this Linas</a>. But that's a whole nutha story. (more intrigue!)<br />
<br />
IBM sales guy: "Remember that system we told you to get rid of?"<br />
<br />
IBM customer: "You mean VM? Sure. We got rid of it, like you told us."<br />
<br />
IBM sales guy: "Well, you're gonna want it back."<br />
<br />
IBM customer: [facepalm]<br />
<br />
<b>Addendum</b><br />
<br />
The 1991 revision of Melinda's paper had a couple of addendums. I picked up printed copies. [remember paper?] One addendum was "Dave Tuttle's Memoirs". This is not usually seen alongside the newer revisions (of Melinda's work).<br />
<br />
At one job, we were hosting lotso Linux on top of z/VM. (That company had never gotten rid of VM anyway.) Trying to inspire my teammates, I made copies of Tuttle's Memoirs and passed them around. [remember printed copies?] Some of the guys actually read it. These were good guys. I don't think they were in it for the money, maybe for the tech. We weren't coding, but I hope they were intrigued.<br />
<br />
Given the myriad search engines available, you can probably find Mr. Tuttle's work for yourself. Here's one I found (other than my paper copy), starting about page 135:<br />
<br />
http://www.cs.tufts.edu/~nr/cs257/archive/melinda-varian/neuvm.pdf<br />
<br />
<br />
Enjoy! Be intrigued. Be inspired.<br />
<br />
Someday I'd like to meet Dave Tuttle in person.<br />
<br />
-- R; <><<br />
<br />
<br />
<br />Rick Trothhttp://www.blogger.com/profile/03475933977328026233noreply@blogger.com2tag:blogger.com,1999:blog-2506532916329743518.post-77395105146944027252014-01-17T05:54:00.000-08:002014-03-07T12:41:51.313-08:00Ho, Ho, Zero<b>Ho, Ho, Zero</b><br />
<br />
Sir Santa tried to stuff the bag with an annual gift or two, but was late. The gifts are also a bit slim this year.<br />
<br />
Of note, <a href="http://vloeistof.casita.net/arc/nord/2013/CD0.iso.gz" target="_blank">CD0</a>, pronounced "see dee zero". This is a rootstrapping tool for <a href="http://www.casita.net/pub/nord/about.html" target="_blank">NORD</a> Linux. The slim part is that the instructions are missing. But the purpose of the thing is to supply a shell and minimal environment. When combined with GCC and BINUTILS (from CD2), the scripts at <a href="http://www.casita.net/pub/nord/build/" target="_blank">http://www.casita.net/pub/nord/build/</a>, and a helping hand from Perl (from CD1), you can "rootstrap" a fresh system.<br />
<br />
This rootstrapping thing (using the current <a href="http://vloeistof.casita.net/arc/nord/2013/CD0.iso.gz" target="_blank">CD0</a>) has been tested on S390 and I386. The CD image also contains support for PPC, which is expected to work just fine, but is untested.<br />
<br />
The other gift in the bag is a minor update to UFT making it leak less info in secure contexts. Seems to work just fine, but we'll have to talk about that in another post.<br />
<br />
-- R; <><<br />
<br />
<br />
<br />Rick Trothhttp://www.blogger.com/profile/03475933977328026233noreply@blogger.com0tag:blogger.com,1999:blog-2506532916329743518.post-26277617258839927902014-01-08T07:33:00.000-08:002014-01-17T06:16:26.462-08:00Climate Change on the Net<b>Climate Change on the Net</b><br />
<br />
There's a storm brewing. Winds of change have shaken loose the low hanging fruit and blown open the barn doors. But fruit which stays on the ground will rot and doors left open will let livestock out and predators in.<br />
<br />
Some points to note ...<br />
<br />
<ul>
<li>IPv6, enabling "the internet of things"</li>
<li>pervasive embedded computing (not your father's Oldsmobile)</li>
<li>cloud computing, futuristic and retro </li>
</ul>
<br />
Even prior to consumer availability of IPv6, we were seeing more "things" on the internet. In my own house, when we replaced our aging Zenith television with a stylish Samsung LCD, we wound up with three "devices" on our home LAN which prior counterparts were not networked. They speak a variety of convenience protocols. Behind my firewall, they're supposedly safe. Supposedly.<br />
<br />
Electronics in vehicles and appliances have increased and gotten steadily more sophisticated. The idiot light says "service required", so you visit your mechanic (or the auto parts store), they plug-in their OBD reader and tell you what's really wrong. Soon, your fridge will tell you to buy milk (or will order milk all by itself). Kinda scary. But also very geeky.<br />
<br />
Cloud computing is the buzzword du jour. It's progressive in that we can buy (rent) disposable complete systems for almost any computing work imaginable. But we deploy complete package bundles, and that doesn't always scale. Default distributions are not by default secure.<br />
<br />
So how's cloud retro?<br />
It takes us back to the "service bureau" days.<br />
That's not a bad thing, just ironic. It reverses the trend over several decades of owning the hardware, but it fulfills part of the MULTICS vision of computing as a utility. <br />
<br />
<b>A Northern Wind</b> <br />
<br />
<br />
NORD is a simple "open source" operating system following <u>the standard recipe</u>. It uses the 'make install' logic defined by individual package authors and supplements that with a ready-to-run scheme. Other than that, it's just Linux.<br />
<br />
Unlike some Linux distributions, NORD tries to be more Unix than Linux. But it is pragmatic enough to use the Linux kernel, GNU core utilities, and other Linux packages.<br />
<br />
The software and computing industry is increasingly distressed about security. (Privacy too, but mostly with the overall hardness against attack.) Consider a recent Wired Magazine article by Bruce Schneier (with help from Jim Gettys, I'm told) ...<br />
<br />
<a href="http://www.wired.com/opinion/2014/01/theres-no-good-way-to-patch-the-internet-of-things-and-thats-a-huge-problem" target="_blank">The Internet of Things is Wildly Insecure</a><br />
<br />
NORD is particularly suited to embedded use. (Like a generic underpinning for things like OpenWrt.) For simplicity, and giving a boost to security, it starts less services and daemons than other systems.<br />
<br />
I use NORD daily for a variety of things, mostly related to software development. (I have a day job.) At one time, NORD's unnamed predecessor served as one of my primary internet-facing servers. It was reliable and secure.<br />
<br />
But NORD needs help. It doesn't have an installer. That's by design, but it needs a way to bootstrap onto new hardware. Someone who can assemble a quick stamp utility for I386 and S390 would really be welcome.<br />
<br />
An old reference doc for NORD is in Google space ...<br />
<br />
<a href="https://docs.google.com/document/d/1QQyCdRe4oHvxFv_23u1DX1rO7EVzhreB6zC3OoCPycM" target="_blank">NORD Scratch Linux</a> <br />
<br />
Recently, friends familiar with the project have lent a hand, one in particular. Their contributions are huge, breaking NORD out of the "one man's toy" mode into something reproducible. Thanks.<br />
<br />
NORD is not the only source based system available. If more people, especially the civic minded hacker brain trust, would use systems like <a href="http://www.linuxfromscratch.org/" target="_blank">LFS</a>, the digital world would be better overall.<br />
<br />
There's a companion project, not so much help on the systems front but a boost to security. <a href="http://casaruso.casita.net/pub/uft/" target="_blank">UFT</a> (Unsolicited File Transfer) has gotten new attention. While its original design was more for interoperability than privacy, it turns out to be great for both goals.<br />
<br />
Join the movement.<br />
<br />
-- R; <><<br />
<br />
<br />
<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://4.bp.blogspot.com/qjr0KIzGsg-qK6n5xnV8W8r5wsWvzkWHtdv3f4gS_y9CPMQW9fA6S8leZL5FBN6J4cihiz7elky5tr7iWmj7NTNPyvHEvn1v_wYMAwlfd55Q-XT0LAc" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="http://4.bp.blogspot.com/qjr0KIzGsg-qK6n5xnV8W8r5wsWvzkWHtdv3f4gS_y9CPMQW9fA6S8leZL5FBN6J4cihiz7elky5tr7iWmj7NTNPyvHEvn1v_wYMAwlfd55Q-XT0LAc" height="213" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">NORD Mascot</td></tr>
</tbody></table>
<br />Rick Trothhttp://www.blogger.com/profile/03475933977328026233noreply@blogger.com0tag:blogger.com,1999:blog-2506532916329743518.post-36785129121717431082013-11-06T17:38:00.002-08:002013-11-07T05:14:27.739-08:00Console - Lose the Password, Reap Better Security<b><span style="font-family: Verdana,sans-serif;">Console - Lose the Password, Reap Better Security</span></b><br />
<span style="font-family: Verdana,sans-serif;"><br /></span>
<span style="font-family: Verdana,sans-serif;">Often, that which seems contrary to what we want is really the best way to get it. </span><br />
<span style="font-family: Verdana,sans-serif;"><br /></span>
<b><span style="font-family: Verdana,sans-serif;">No More Passwords</span></b><br />
<span style="font-family: Verdana,sans-serif;"><br /></span>
<span style="font-family: Verdana,sans-serif;">You may have heard talk about being more secure by not using passwords. Specifically, if we use means other than traditional password access then the target systems and services may be better defended. In English, using a PKI certificate, an SSH key, or some other form of credentials is typically more secure than ye olde username/password pair. </span><br />
<span style="font-family: Verdana,sans-serif;"><br /></span>
<span style="font-family: Verdana,sans-serif;">I agree with this view. </span><br />
<span style="font-family: Verdana,sans-serif;">For many of my systems, there is no password. </span><br />
<span style="font-family: Verdana,sans-serif;">I don't mean simply that the password is unknown. I certainly don't mean that the password is blank. What I'm saying is that there is <i>no usable password</i>. Anything typed in will fail. To access these boxes, the user (typically moi) must take one of the other routes. </span><br />
<span style="font-family: Verdana,sans-serif;">It works. </span><br />
<br />
<span style="font-family: Verdana,sans-serif;"><b>Unusable Passwords </b></span><br />
<br />
<span style="font-family: Verdana,sans-serif;">A quick way to render root's password unusable is to code an asterisk in the password field (either <span style="font-family: "Courier New",Courier,monospace;">/etc/passwd</span> then '<span style="font-family: "Courier New",Courier,monospace;">pwconv</span>' or <span style="font-family: "Courier New",Courier,monospace;">/etc/shadow</span> directly). This is the norm for service accounts (bin, daemon, mail, nobody, ftp). Best practice is to employ '<span style="font-family: "Courier New",Courier,monospace;">sudo</span>' for all root work. Sign on as yourself (with SSH or whatever), then '<span style="font-family: "Courier New",Courier,monospace;">sudo</span>' as needed. There's an audit trail. How novel. </span><br />
<br />
<span style="font-family: Verdana,sans-serif;">So ... you don't really need a root password anyway, now do you? </span><br />
<span style="font-family: Verdana,sans-serif;"><br /></span>
<span style="font-family: Verdana,sans-serif;">But I propose something even more radical. </span><br />
<span style="font-family: Verdana,sans-serif;"><br /></span>
<span style="font-family: Verdana,sans-serif;">I wear the sysadmin hat daily. Whether for development or for hobby and home, I get to play "root". (Some of us actually enjoy this, in small doses.) </span><br />
<span style="font-family: Verdana,sans-serif;"><br /></span>
<span style="font-family: Verdana,sans-serif;">Virtualization is virtually everywhere. Cool! </span><br />
<span style="font-family: Verdana,sans-serif;">But one of the most annoying things about virtualization is the double-signon effect. In my sysadmin mode, I get confronted with this regularly. I'm signed onto the host, but the guest throws a password prompt. [sigh] Not surprising; makes sense; but is in the way of real work. </span><br />
<span style="font-family: Verdana,sans-serif;"><br /></span>
<span style="font-family: Verdana,sans-serif;">It's worse than "in the way", more than a hassle. For the guest to process your secondary sign-on, it must have ... drum roll ... a <i>password</i>. Now wait just a minute. We're trying to do away with passwords!! Those of us trying to evolve beyond passwords find this situation positively primordial. (a prime ordeal) What to do?? </span><br />
<span style="font-family: Verdana,sans-serif;"><br /></span>
<span style="font-family: Verdana,sans-serif;"><b>No More Login</b> </span><br />
<span style="font-family: Verdana,sans-serif;"></span><br />
<span style="font-family: Verdana,sans-serif;">The modest proposal: throw a shell on the guest console directly. </span><br />
<span style="font-family: Verdana,sans-serif;"><br /></span>
<span style="font-family: Verdana,sans-serif;">Shocked? You should be. (Unless you've heard it before.) </span><br />
<span style="font-family: Verdana,sans-serif;"><br /></span>
<span style="font-family: Verdana,sans-serif;">It's not a new idea. Some of us have been recommending this for several years. Do this in concert with unusable passwords. Replace '<span style="font-family: "Courier New",Courier,monospace;">getty</span>' with a shell. (The security guys usually don't "get it" about the nature of '<span style="font-family: "Courier New",Courier,monospace;">getty</span>' so they throw a hissy fit and we sysadmins are forced to acquiesce.) But it's a good idea, and in the long run <i>more</i> secure (for virtual consoles) than a login prompt. </span><br />
<br />
<span style="font-family: Verdana,sans-serif;"></span>
<span style="font-family: Verdana,sans-serif;">With an operational shell, not a login prompt, you get immediately to work. There is no fumbling around at the guest console. There is no password to be concerned about. </span><br />
<span style="font-family: Verdana,sans-serif;"><br /></span>
<span style="font-family: Verdana,sans-serif;">I wish the security guys (or any objectors) would follow this line of reasoning. Someone accessing a virtual console has already been vetted by the host. Someone with control of a guest (even if a password prompt is in effect) can do so much more, whether good or bad. So if they #1 have been verified and #2 are in a position to completely re-image the guest, then a no-login-required shell is perfectly reasonable. Combine that with the unusable passwords trick and your virtual systems are hardened on one more front. </span><br />
<br />
<span style="font-family: Verdana,sans-serif;">Depending on the physical security of
your data center, this crazy concept might be a good idea for physical
consoles too. The security issue is the same: you're sitting in front of a physical machine with full ability to reboot, re-image, anything. We presume that your data center is not the public library, where password-protected consoles still make sense even though operators freely Ctrl-Alt-Del. Results not typical. Use with caution. Your mileage may vary. Do not fold, spindle or mutilate. And I am not a lawyer. </span><br />
<br />
<span style="font-family: Verdana,sans-serif;">If your shop is small, then you may *be* the security guy. You could schedule a meeting with yourself, discuss the pros and cons, and convince yourself that this is in fact a good idea. (But talking to yourself is a red flag for anyone in that role.) </span><br />
<br />
<b><span style="font-family: Verdana,sans-serif;">For Example</span></b><br />
<span style="font-family: Verdana,sans-serif;"><br /></span>
<span style="font-family: Verdana,sans-serif;">For most of my guest systems, I replace 'getty' with a no-login-required script of some sort. The contents of <span style="font-family: "Courier New",Courier,monospace;">/sbin/conshell</span> are roughly as follows. </span><br />
<br />
<br />
<blockquote>
<span style="font-family: Verdana,sans-serif;"><span style="font-family: "Courier New",Courier,monospace;">CON=console<br />if [ ! -z "$1" ] ; then CON="$1" ; fi<br />if [ "$CON" = `basename $CON` ] ; then CON=/dev/$CON ; fi<br />PS1='\$ ' ; export PS1<br />exec sh -i 0<$CON 1>$CON 2>$CON</span> </span>
</blockquote>
<br />
<br />
<span style="font-family: Verdana,sans-serif;">Yeah yeah ... there are spiffier ways to code the conditionals. (Where is Jon Miller when I need him?) I've left this as-is because it works on a wide range of shells. </span><br />
<br />
<span style="font-family: Verdana,sans-serif;">Thanks for reading. Stay safe. </span><br />
<span style="font-family: Verdana,sans-serif;"></span><br />
<br />
<span style="font-family: Verdana,sans-serif;">-- R; <><</span><br />
<br />
<br />
<span style="font-family: Verdana,sans-serif;"><br /></span>Rick Trothhttp://www.blogger.com/profile/03475933977328026233noreply@blogger.com1tag:blogger.com,1999:blog-2506532916329743518.post-58690011220175581862013-10-18T12:59:00.000-07:002013-10-18T13:01:19.009-07:00Lessons 67 and 68 - Internet OutageLessons 67 and 68<br />
<br />
So far, I've lost 8 hours and $150. But we have internet again. <br />
<br />
It was Tuesday. I am normally at the office on Tuesdays. But my teammate was out and I have a full plate, so I thought I'd save the commute time and work from home. While sitting with my wife and contemplating the day before us, I noticed the lights blink. We both heard a beep. Right away I knew two or three appliances had reset, including a server or two. No biggie, so I thought. <br />
<br />
When we have an outage, I try to learn from the mistakes. Whether I missed a step or some system has let me down, it's a healthy challenge to review and adjust. <br />
<br />
<b>Mistake Number 1</b> - delayed UPS battery replacement<br />
<br />
The beep was from the UPS which covers our "important" computer gear: <br />
the cablemodem, the router, and the main server. The BSL1079 (lead/acid "gel cell") failed long ago, but I had been using an aging car battery. This was not simply putting things off. The car battery, even aging, has more than ten times the capacity of the normal UPS battery. But either the battery had aged more than I knew or my spit-n-bailin-wire rigging had loosened up. This was totally my fault.<br />
<br />
<b>Lesson:</b> just order the [expletive deleted] normal battery and do high capacity as a separate project. <br />
<br />
<b>Mistake Number 2</b> - mixing services ... and service levels<br />
<br />
Some time back, all our stuff hubbed off a server called "main". <br />
NFS, YP/NIS, SMB, NTP, DNS, SMTP, IMAP, internal HTTP, and notably DHCP. Most of these services have been doled out to dedicated appliances or to service providers. The exception is DHCP. So when the primary server powered up, it had these old filesystems to check. (Things did not come down clean, so an integrity check is warranted.) DHCP had to wait until that was done. <br />
<br />
The filesystems are still used, but with a lower service level requirement. DHCP has a much higher service level requirement, especially with increased WiFi. So the idea that a high priority service is waiting behind a lower priority service is bass-ackards (as we say in Texas). This will change. My fault, there's history. <br />
<br />
<b>Lesson:</b> consider service requirements and plan accordingly. (DHCP will move)<br />
<br />
<b>Mistake Number 3</b> (not mine) - deceptive diagnostics (and this was the worst)<br />
<br />
With the server back, and the network units functioning normally (I thought), I checked on our IPv6 tunnel server. This is a Xen virtual machine hosted by the same physical box as "main". Native IPv6 is not available yet where we are, but the SixXS tunnel does nicely. But the tunnel wasn't starting. <br />
<br />
Turned out that IPv4 connectivity was still down. <br />
Turned out that the router had no DHCP lease from our ISP. <br />
After multiple (controlled) on/off cycles of both the cablemodem and router, the relationship was still "we're not talking". Activity lights, yes, but all zeros for the external address. Plugged in a laptop directly to the cablemodem; got a lease! So that indicated clearly the router has failed. Clearly. <br />
<br />
This NetGear router has been giving us a little trouble on the WiFi side. Dunno if it is just RF interference or perhaps something we can blame on the internet provider. There are gaps in 802.11 coverage inside the house, dead zones. So off I went to Best Buy, returning with a shiny new LinkSys "AC" model, with A/B/G/N backward compatibility. <br />
<br />
The new router also failed to obtain a DHCP lease. Huh?!?<br />
<br />
Pause and reflect: Old router, no lease. Laptop, yes lease. <br />
But new router, also no lease. The cablemodem is just not smart enough (one would think) to distinguish between a "computer" and a "router". I had already tried cloning the MAC address of the laptop to the old router so it would look to the provider's DHCP server "just like the laptop" (which had succeeded in DHCP). No joy. <br />
<br />
This is where I lost the rest of the day ... multiple reboots, power on/off cycles, WiFi reassociations, and DHCP transactions (on the "inside"). Cablemodem worked with two computers, failed on one, failed on both routers. The new router got a DHCP lease on our internal LAN, and then so did the old router. [sigh] (I could have gotten a better priced on the new router if I were not in a rush from the outage.) <br />
<br />
What a waste of time. <br />
<br />
What finally worked was to put our old "firewall" on the cablemodem. This is a Linux box with two ethernet ports. That's one I got right. And the "mistake" was misleading cues from Time/Warner Cable's device. <br />
<br />
<b>Lesson:</b> hang onto what worked before, at least one generation.Maybe consider changing internet providers! <br />
<br />
<b>Mistake Number 4</b> - too much reliance on internet (?)<br />
<br />
Uhh... maybe not. <br />
We do *business* online. <br />
For most transactions, using the Internet is no less legitimate, even more reliable, than using the telephone. So what are you gonna tell me? Too much reliance on the phone? <br />
<br />
I grant you that if we rely on internet for these things we need to have <u>reliable</u> <u>alternatives</u>, whether a procedural habit fall-back to voice or perhaps redundant internet service. Yeah ... that's it. Redundant internet service. In a prior job, my employer paid for internet and we paid for a second line, keeping personal and "work" on separate channels. WHEN (not if) there was a failure on one, I would switch all traffic to the other. <br />
<br />
The idea that Joe Suburbia have two internet contracts is silly. <br />
The idea that my family have two internet contracts is less so, but still pricey. <br />
<br />
<b>Lesson:</b> be prepared, and have alternatives.<br />
<br />
I wrote this in a hurry because I thought it should be dispatched quickly. Hopefully it's not too jumbled. Hopefully your internet experience is more effective. <br />
<br />
-- R; <><<br />
<br />
<br />Rick Trothhttp://www.blogger.com/profile/03475933977328026233noreply@blogger.com0