I’ve hacked a simple script for drawing a file system tree as a graph with graphviz. Sounds fun?

Once I realised than a file system tree is like a MindMap…

Here’s the gitlab link: https://gitlab.com/Lockywolf/scsh-xattr-mindmap

Contrary to the name, it is actually in Chibi, not in scsh. I initially thought that scsh would be better due to more exensive posix support, but it turned out to be that Chibi was good enough.

It is a small-ish (500 lines of code) script to generate a graph from your filesystem tree. It accepts a few options (editable directly at the file top) and duplicates quite a lot of the GNU Find functionality, but I didn’t find a way to avoid doing that, as it has to use heuristics in order to prune the tree to a reasonable size.

The resulting image is like this:

Small, 1Mb

Large, 44Mb

I plotted the Slarm64 (Unofficial Slackware for Raspberry Pi) repository tree, just for the demonstration purposes.

The size of the images above is 1×2.5 metres. It’s large, but my original goal was to plot my whole file system. The ’size=’ parameter is tunable. I think it is reasonable to assume that you need to have at least 4 square centimetres per node, so a graph that large would accommodate about 4000 nodes. In my opinion, 8000 is still possible, but too tight.

With the default settings the script ignores regular files, but traverses symlinks. In theory it also supports hardlinks, but you would need to turn on drawing regular files manually.

I made this script, because I started to feel that I am starting to forget what I have on my hard drive, that has amassed quite a lot of life history for the past 20 years. (Since hard drives became reasonably priced.)

Use-cases and pull requests welcome. One more reason to create this script was to prove that Scheme can be a practical programming language.

Technologically, this code is not terribly advanced, the only trick that may be interesting to programming nerds is having the r7rs module and the main function in the same file (like scsh/scheme48 suggest doing), which requires procedural module analysis.

I had to glue on a couple of C bindings for sys/xattr.h, those are now available at the Snow-Fort repo. Those are Chibi-specific.

Hope you will enjoy it.

Solving SICP

This report is written as a post-mortem of a project that has, perhaps, been the author’s most extensive personal project: creating a complete and comprehensive solution to one of the most famous programming problem sets in the modern computer science curriculum “Structure and Interpretation of Computer Programs”, by Abelson, Sussman, and Sussman (\cite{Abelson1996}).

It measures exactly:

• How much effort SICP requires (729 hours 19 minutes (over eight months), 292 sessions).
• How many computer languages it involves (6).
• How many pieces of software are required (9).
• How much communication with peers is needed.

It suggests:

• A practical software-supported task management procedure for solving coursework.
• Several improvements, on the technical side, to any hard skills teaching process.
• Several improvements, on the social side, to any kind of teaching process.

The solution is published online (the source code and pdf file):

• http://gitlab.com/Lockywolf/chibi-sicp (\cite{chibi-sicp})

This report (and the data in the appendix) can be applied immediately as:

• A single-point estimate of the SICP problem set difficulty.
• A class handout aimed at increasing students’ motivation to study.
• A data source for a study of learning patterns among adult professionals aiming for continuing education.
• An “almost ready” protocol for a convenient problem-set solution procedure, which produces artefacts that can be later used as a student portfolio.
• An “almost ready”, and “almost convenient” protocol for measuring time consumption of almost any problem set expressible in a digital form.

Additionally, a time-tracking data analysis can be reproduced interactively in the org-mode version of this report. (See: Appendix: Emacs Lisp code for data analysis)

Let us discuss SRFI-216: SICP Prerequisites

Abstract

This post is a not-so-technical introduction to the Scheme Request for Implementation 216: SICP Prerequisites, that I have written and made available for discussion. (Please, contribute!)

SICP stands for the “Structure and Interpretation of Computer Programs”, a world-famous introductory programming textbook.

It’s aim is to make the exercises and examples from the book be available for any Scheme system bothering to provide it, and not just MIT/GNU Scheme and Racket (which doesn’t even consider itself a Scheme any more). Before this SRFI an issue tracker request asking for SICP support would have been looking vaguely. Now you can just write “Could you consider providing SRFI-216 (and 203)” in your implementation.

In order to write this SRFI, I went through the whole book and solved all the exercises. However, my experience is just mine, and to make a truly good common vocabulary, community feedback is required.

For technical detail and more background, I am inviting you to read the whole article.

A review on “Psychology of Intelligence Analysis by Richards J. Heuer Jr.”

Abstract

Richards J.Heuer Jr.is one of the people who revolutionised the way intelligence content is produced in the Central Intelligence Agency of the U.S.A. Speaking crudely, his main contribution was the introduction of the “Scientific Method” into the everyday routines of the CIA analysts. This book is partly his self-reflection on this transformation, and partly a list of heuristics that any intellectual worker could employ to improve his own efficiency (and self-satisfaction). I found it very good. It clarified quite a bit of concepts I had been only vaguely aware of, and helped me hone a few of my own ideas.

I actually recommend reading it to everyone, and perhaps would even suggest studying it at school, because it is hard to find a skill of more generality than a skill of thinking. And the intelligence aroma just makes the book more exciting for kids.

If you are interested in more detail, welcome under the cut.

Proposing programming language features

This blog hasn’t had enough attention for quite a while. This is not, however, because I have abandoned it, but rather because the original purpose of this blog, that is dumping essays regarding books I read, is still valid. It’s just that the most recent book has taken an order of magnitude more time than I had expected it to take.

Okay, I’m going to write a bigger and better review on the “Structure and Interpretation of Computer Programs”, but the book altogether took so much time, effort and emotions, that even writing the review is going to take a while.

Meanwhile, one of the by-products of my reading happened to be a proposal of a feature to be included into the Scheme Language, that is needed in order to support all the code examples in the book.

(Yes-yes, you’re not misreading it. The book published in 1996 is _still_ not covered by the existing language standard in full. Otoh, it means that there is a chance of achieving things.)

Now that the proposal has an official number, there is going to be a public discussion among the potential Language System providers, and maybe (if it passes the review), we will have this feature officially recognised.

Watching discussions of experts on what they may actually work themselves is a fascinating experience, and a chance to improve own skills too. In this case the discussion is not expected to be too heated, however, but anyway.

https://srfi.schemers.org/srfi-203/ — this is the link to my recent proposal.

It speaks about quite an interesting approach to generating computer images, that builds on top of the classical features present in most drawing languages, such as PostScript, TikZ, MetaPost or SVG. While the expressive power is largely the same, the degree of abstractness is greater, which leaves gives greater code reusability and flexibility.

Scheme is by not means the only language that has a community feature review process.

• Python has Python Enhancement Proposals (PEP)
• Java has Java Community Process (JCP)
• Scheme has Scheme Requests For Implementation

The image in the header is a digital copy of a work of M. Escher, whose works have inspired the original author of the “Picture Language” Peter Henderson. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.137.1503

A Review for GNU Autotools by John Calcote (1st. ed.)

/ I saw a book titled ≪Die GNU Autotools≫ and I thought “My feelings exactly”. Turns out the book was in German. – Tim Martin./

I have read the GNU Autotools: A Practitioner’s Guide by John Calcote.

This is a small review of mine. This time the review is really small, mainly because there already is a good review, written by Flameeyes  https://flameeyes.blog/2010/08/20/book-review-autotools-by-john-calcote/, and even though ther review was arranged by the original publisher, No Starch Press , it is technically accurate and psychologically unbiased.

What I do have to add are just a few of my own experiences.

Russian kids are not taught a proper lesson in computer information exchange.

While this statement is a bit radical, most often there is not a single consice introductory lecture which would teach people how to make descent programs, and saying broader, software pakages out of scattered and disperse algorithms, which people are taught to create in high school and in universities.

Most people learn this by either struggling with Microsoft’s documentation, or (even worse) by copying existing Free Software projects, and tweaking the scripts.

This is a very bad approach.

I know, this is a very emotional statement. But my experience says the following: the less structured is the field you are studying, the more structure needs to be present withing the courses given.

As an example: C, C++, HTML, are reasonably well-structured topics. No matter how you approach the studying, there is only one definitive standard of those, and you have almost no chance of completing your tasks successfully unless you comply with the rules.

As an opposite example, creative writing is not that structured. Your literature teacher may give you good grades, or bad grades, but that would quite often be affected by his personal properties rather than his experiences with the audience you are targeting. In case of the high school class essays this doesn’t matter so much, because your target audience IS your teacher, but it gives you little expertise on how to write for other people.

Therefore, creative writing teaching should require much more elaborate courses than computer programming, exactly to overcome this uncertainty.

Making software packages is an ill-defined task.

You cannot ever solve it precisely. You can reach some level of closeness to the original goal, but as life evolves, your projects have no choice but to adapt to the changing world. Therefore creating software packages is a badly structured task

This makes it necessary to have a structured course. Calcote’s book provides such a course.

Indeed, despite being called “A practitioner’s guide.”, it is more of a good theory treatise than a cookbook. And this is good, not bad. Nowadays canned recipies are in abundace, at StackOverflow, for example. But well-narrated explanations are as rare as they have ever been.

Shall this book be studied at school.

No. Of course not. But it would be a very good idea to let university freshmen read this book as a part of their ‘practice of programming’ introductory classes.

Statistics

This book is 315 pages long, and required 29 hours 54 minutes of my time, including doing exersises and writing this review. This makes it roughly 10 pages per hour, 6 minutes per page. Those were produced in 11 study sessions, each session 2 to 4 hours, and was accomplished in one week, with more than one session per day on the weekend. The book was printed out on paper, and I was making annotations with a pencil. For practical text writing I was using GNU Emacs 26.2. In the

Bibliography

• Autotools Mythbuster
• Die GNU Autotools by Florian Stöhr
• No Starch Press’ webpage for the preview of the second edition
• GNU Website for Autoconf

Review for SSH Mastery by Michael W Lucas

Cover

Motivation

Why did I decide to read the SSH Mastery?

I have been using SSH for a long time, learning it bit by bit, intuitively. At some point I decided to arrange my knowledge in a more structured shape. So at that time I had a choice, whether to read official SSH documentation, or to look out for some external information, preferably from people who have experience of not just creating the product, but also of using it.

The problem with more prosaic books though is that they often risk just retelling the original documentation verbatim, which is always the safest choice, as all the errors may then be attributed to the software authors. On the other hand, they have a risk of falling into a terribly newbie- oriented, very verbose narration style, which would make them too voluminous for being digested in observable time.

Michael W Lucas’ book skilfully manages to avoid both of those pitfalls. It’s both concise and easily readable. Therefore, it didn’t take me too long time to print it out and dive into.

Why a book on SSH is even needed?

Indeed, why? What SSH even is? SSH stands for Secure SHell, but it actually has nothing to do with shells or cores. A ‘shell’ is just a name, traditionally used in computing to denote a way to ask a computer to perform exact tasks. That is, 90% of exact tasks are asking a computer to compute something. You know, that mode of your computer, when this huge 1000$ machine actually works as something it was designed to be 70 years ago: a huge calculator. (By inexact tasks I usually denote tasks which require automated, but not exactly computational tasks, such as calling your friends by a voice chat. )

When you use your calculator, you don’t really need any specific introduction into typing numbers, do you? Well, as I said, a personal computer is a very huge, and a very advanced calculator, so it needs a special advanced program to ask you for your formulae.

The word Secure here implies that using SSH, you can actually ask other computers to be the calculators you ask to compute something for you, and those only need to have the SSH on them, and be connected to a computer network.

So the main task of SSH is establishing connections to other calculators.

Is it so hard? Why would anyone have an extensive manual for a program which performs such a simple task?

There are many answers, but the main one is: not being obvious. The hardest thing to get used to when working with computers is that they are actually one of the least obvious things you find in this world, even less obvious than people.

It doesn’t mean that they are hard. Quite on the opposite, since computers are very dumb, and quite fast, if a good manual is available, few things are actually easier than computers.

But computers are painfully unobvious. Which image among the ones you see on your screen is a real one, and which one is a compressed one? No way to tell. Is the password you are typing on a website stored somewhere or is it not? Are you being attacked by hackers right at this moment, when you are reading this?

The answer to the last question is ‘yes’, even if you are not even remotely aware of the existence of those hackers. And they are also not even remotely aware about your existence, but they don’t even need to. They have robots do all the dirty work for them.

And this is also the place where official manuals fail to increase the obviousness of what is going on, as they are bound to only describing what a program should do, but not what the program should NOT do.

What is the book about?

SSH is usually used for connecting to a remote computer and giving it some orders using the command line.

One of the reasons for reading the book is learning that SSH can to much more than that. As I already said, it is very good at making any soft of connections between two machines.

A typical example: an SSH can work as a universal (so-called SOCKS5) proxy server between you browser and some computer in a country with a less outrageous censorship law. (Yes, to go around those nasty screens telling you that the government decided to block your access to some website.)

Or, SSH is able to forward just one connection from your computer to another computer. This may be very useful if your company is restricting websites which you are able to visit.

Yes, although this post is NOT about circumventing censorship, for non system administration people it’s not obvious (another picture on “not obvious”) that going around barriers is the very nature of computers.

Or, SSH is able to forward everything leaving your computer through a computer far away. Can be very useful if you company’s system administrator is collecting your bank card numbers for his own pleasure.

SSH can simplify a lot of cryptography for you, and the book is very practical about how to make your life as painless as possible while working with cryptography, and (if you never tried), cryptography is still a very very meticulous task, where every little bit helps.

Writing style

The SSH Mastery book is good at simply and in a concise manner explaining the (quite complicated) things. To be honest, I was astonished to see how many practical details Michael W Lucas managed to include in his book, and still keep it under 200 pages.

The book is enriched with practical examples from his own working experience, and he, besides being a writer, is one of the world’s best experts on secure operating systems. (He also wrote the Absolute BSD book.)

The book is also written in a swift, light, vocabulary-rich language. (Which is less surprising than it may seem, as Michael W Lucas is also writing fiction books(!).)

One of the best properties of the book is that it gives you the right tools to reach that sweet feeling of almightiness, which is one of the main reasons why people are choosing computers as a career. And this book makes this feeling not limited to just computing (or systems administration) professionals.

The book collection

Michael W Lucas is writing a series of books actually, under the shared name of ‘IT Mastery’. In this series he elaborates on those tools that systems administrators are learning by trial and error, and users often completely ignore, even though these are those tools which are actually controlling their lives and are used every day. All those have practical examples and accompanying life experience stories.

Should everyone read this book? Shall it be taught at school?

Well, I am sad to say no. Indeed, most people, most developers, and even most systems administrators can get by without ever consulting any reference material on SSH, and especially a prosaic book.

But saying that, most people can get by without so many things… Maybe it’s sometimes better to think in terms of enriching life, rather than surviving?

P.S.

Michael W. Lucas has no relation whatsoever to the film director Michael Lucas.

2019-07-04T10:25:13, Shanghai