KISS – A computer odyssey

Introduction

In a recent post on misc@openbsd.org Ingo Schwarze, an OpenBSD developer, wrote as follows.

I waste time whenever i have to select anything from any kind of menu, select any icon from any kind of iconbox or desktop background, select any file or directory from any file selection dialogue, or have to click any icon in any dialogue box.

I prefer typing commands and use as little menus, clickable icons, selection lists, and dialogue boxes as possible because it is faster, simpler, and requires less looking at the screen.

My feeling exactly. I favour the KISS (Keep It Simple, Stupid) principle in computing, although it took me a long time to realise this fully. But looking back I see it’s a continuing trend.

MS-DOS to Linux

I got my first PC some time in the early 1990s to replace an Amstrad PCW, if anyone remembers that. The PC came with MS-DOS and Windows 3.0, which was replaced almost immediately by Windows 3.1. I didn’t upgrade. I didn’t see much point in Windows and continued to use DOS until, at some point in the 1990s, I began to experiment with Linux. After an initial unsuccessful attempt to use Slackware I tried Red Hat and got on better with that.

I knew nothing about Unix at this stage; in fact, I still knew very little about computers in general. But I was beginning to enjoy the intellectual challenge they offered and that was what drew me to Linux. At this time it was pretty much entirely reliant on the command line, but this wasn’t a problem for me because I was used to it from DOS and in fact preferred it. But the lack of word processors was a different matter.

Linux and word processors

My practical reason for using a computer was as a writing tool for books and articles. In my Amstrad and DOS days I’d had a word processor called Protext which I liked, but it wasn’t available for Linux. In fact, there weren’t any real word processors for Linux at the time; you pretty much had to use either Emacs or Vim. I tried both and eventually settled on Vim. Some people disliked it because it is modal (you have to alternate between Insert mode to write the text and Normal mode to modify it), but that didn’t worry me because Protext is also modal.

I still use Vim all the time, even though there are now plenty of word processors and similar programs available to me. The only one I have any use for is LyX, which I’ve used to produce seven books, but even there I usually write my text first in Vim and then paste it into LyX for final editing before printing.

Desktop environments and window managers

Together with the switch to Linux I also began to use the graphics mode, and that led me to explore the world of desktop environments. KDE and Gnome intrigued me at first but gradually I realised that I didn’t need or want either of them. Eventually I settled for a simple stacking window manager, IceWM, and was happy with that for a long time, until I tried tiling window managers. Since then I’ve been sold on them; nothing would persuade me to go back to a stacking WM. My favouring tiling WM, as I’ve explained elsewhere, is Spectrwm. (See my previous blog for details.)

From Linux to OpenBSD

I’d long had a curiosity about the Unix philosophy and I tried out FreeBSD, a couple of times, but found it rather frustrating and always went back to Linux. Then I tried OpenBSD, simply from curiosity. Over a period of about six months I became completely hooked, and four years later I still am.

The transition from Linux to OpenBSD was probably easier for me because the Linux distribution I was coming from was Arch, which is often said to be the most Unix-like version of Linux. I very much doubt I shall change my operating system again. Ths Unix philosophy  is based on KISS and that’s what suits me.

Conclusion

Looking back now over almost 40 years of messing about with computers (and doing some proper work with them in the process, obviously!) I see that there has been a gradual but quite definite progression towards simplification and minimalism.  I know that at least some other people have had a similar experience. If that’s your case I hope you may find these jottings of interest.

A taste for `minimalism in computing probably reflects a more general attitude to other things in life. I’ve written a post about that as well.

How to make a sitemap.xml for a medium-sized site & edit it with Vim

Introduction

In this post I describe two topics in the creation of xml sitemaps:
1.  producing the sitemap.xml file and 2. editing this file to correct errors.

1. Problems with creating xml sitemaps

If you want a sitemap.xml on your website there are places on the Net where you can make one for free, but most of them have restrictions of one kind or another. They all, as far as I know, limit your pages, often to something like 500 or fewer. That’s OK if you don’t have too many, but I have more than 3500.

Perhaps you don’t need to index the whole site, or may prefer to split it and index different parts of it separately. This may get round restrictions on the number of pages you can index and will certainly be quicker. But it requires you to start indexing at a subdirectory below your root directory and some sitemap generators don’t allow  this.

Currently I’m using Online XML Site Generator. It’s free (though you are invited to make a small donation  – £2, equating to 2.46 USD or 2.34 EUR, which seems very fair). The number of pages crawled is limited to 2000. Here is a full list of the conditions.

  • Maximum pages 2000 (includes pages which errored)
  • Spider will run for maximum of 60 minutes.
  • Individual page timeout 20 seconds
  • Download limit of 60K (for larger files the first 60K is downloaded).
  • Maximum of 20% urls hitting 20s second timeout.
  • Average request time across all urls must be 75% of timeout (which is 15 seconds) sampled every 25 urls.

As you are allowed to start at a subdirectory I can live with these conditions, including the 2000-page limit; I make separate sitemaps for different areas on my site.

However, I find that the sitemaps so produced need editing. They are not accurate; they contain links to all the pages in the relevant subdirectory but include others as well.

To see how this works, suppose you have a website called myfruit.com, with subdirectories myfruit.com/apples, myfruit.com/oranges, and myfruit.com/pears. You make a sitemap for myfruit.com/apples. This should contain URLs for apples, but in fact it will contain many URLs from the other subdirectories.

2. Editing the sitemap.xml file

You can easily edit the sitemap.xml you’ve just created, but if you have to remove a lot of unwanted entries manually it will take you a long time. Fortunately Vim (the One Editor to Rule Them All) allows you to automate the process almost completely. (Yes, I know other editors are available but I don’t use them, so if that’s what you use you’ll have to work out your own method.)

Taking the example website above, let’s suppose you have made a sitemap called sitemap-apples.xml. When you load this in Vim you see that as well as entries for apples there are also lots of entries for oranges and pears. They will look like this.

<url>
<loc>https://myfruit.com/pears/</loc>
</url>

To remove all the entries containing “pears”, use  Vim’s macro (recording) facility. Here are the keystrokes,with comments (after #).

Start in Normal mode and issue these commands.

  • qa   # start recording and store results in register a
  • /pears   # find first line containing “pears”
  • k   #  move cursor up one line
  • 3dd    # delete three lines
  • n   # find next instance of “pears”
  • q  # end recording

You have now set up your macro. Test it by doing @a. This should delete one instance of “pears” and position your cursor on the next instance. You can repeate it with @@.

If this works, you can continue to delete all the remaining instances of “pears” automatically by doing something like 300@q. Don’t worry if this gives more repetions than necessary; these will simply be ignored.

Make a similar macro for “oranges” and you should be left with a correct sitemamp.xml file that you can upload to your site.

3. Saving the macros

If you only have one or two sets of unwanted URLs to delete it won’t take long to type the macros. But what if there are more than this, or if you update sitemap.xml frequently? Can you save your macros for future use? Yes, you can, and it’s easy.

There is a site which provides most of the information you need, but I found it didn’t work exactly as specified, possibly because of later changes to Vim. Here is what I do, which works as of 12 May 2019.

  1.  Add lines like these to your .vimrc.
    let @a=’/pears^Mk3ddn’
    let @b=’ /oranges^Mk3ddn’

NB.  ^M is produced in Vim by typing Ctrl-v <Return>.

2. Load sitemap.xml into Vim. Do 200@a. This should delete 200 instances of pears. Now do 200@b; this should do the same for 200 instancs of oranges. (Don’t forget to source .vimrrc or stop and reload sitemap.xml in Vim after you’ve modified .vimrc.)

 

 

 

 

 

 

 

 

How to avoid problems with Vim undo & redo

Starting with Vim 7.0 this brilliant editor introduced a more powerful set of commands for Undo and Redo, using an undo tree. Being lazy, I didn’t fully get to grips with this and as a result didn’t really understand what I was doing. At times I accidentally deleted text I’d just writtenjjjk and then couldn’t recover it.

I knew the previous version of my work had to be there but I didn’t know how to get it back. After a time I realised that when things seemed to go wrong the solution was to use ‘g-‘ and ‘g+’ to go hack and forth in the undo tree.  But this was a bit hit & miss.

If you have had a similar experience here are a couple of useful site where things are explained clearly;

https://vim.fandom.com/wiki/Undo_and_Redo.

https://vim.fandom.com/wiki/Using_undo_branches

The second of these sites has links to a couple of plugins which make life easier. I’m not generally an enthusiast for plugins but I’ve installed Histwin, which seems pretty good; I’ll probably try Undotree as well.

Finally, thanks to a third page I now know why I occasionally deleted work I’d just done and couldn’t recover it. See:

https://vim.fandom.com/wiki/Recover_from_accidental_Ctrl-U

This explains why it happens (pressing Ctrl-u in Insert mode) and provides a way to avoid it.

 

 

 

How to save $HOME when reinstalling OpenBSD

Everything has gone wrong and you have to reinstall from scratch. It shouldn’t happen but it may if you do something silly – I once acccidentally deleted /etc! But you have all sorts of stuff in $HOME that you don’t want to lose. Can you preserve it?

Well, of course, you should have a backup. I save one nightly on Tarsnap. But it may take a long time to restore from backup and it would be better if you didn’t have to.

Fortunately you can save your $HOME quite easily. Here’s how.

  1. When the Install process asks you to use disklable, choose either OpenBSD or Whole Disk.

2. See which partition contains /home. Suppose it is ‘k’. Type ‘m -k’ (without the quotes). Choose ‘No mount point’.

Complete the Install process. When the system restarts,  edit /etc/fstab to mount the ‘k’ partition as /home.

It should work – it has worked for me. But if at all possible make a backup first, just in case it doesn’t.

 

 

An eccentric method of writing blog articles?

When writing posts or articles that are to appear on web pages using programs such as LyX or WordPress it probably seems natural to simply start doing so directly, using the native editor of the program in question. I do this this if I’m writing a fairly short piece, perhaps one or two paragraphs. But if it’s a longer or more complicated text I prefer to do it in a more roundabout way, starting in Vim and pasting the material into LyX, WordPress or whatever later. (The exception is when I’m composing emails, but that’s because I have Mutt set up to use Vim as editor.)

This may seem a little eccentric but I’ve always found it makes my life easier. I’ve been using Vim for many years and it’s mostly second nature to me now. I can think as I’m writing, make quick changes – delete, add, move words, sentences, whole paragraphs – easily and quickly. I can instantly reverse changes I’ve made and undo those changes instantly as well. At the end I can do a spell check to look for typos.

The standard editors supplied by  the programs allow most of this too, of course, but a good deal less intuitively.  It’s more difficult to separate the words you writing from you are writing from how they are going to appear on screen or on the page. But these are different  things.

Writers are often asked whether they write in longhand or directly on the keyboard. People who write in longhand often say that physically shaping the words gives them a sense of creativity and immediacy that is missing if they type at the keyboard. It’s tempting to think that there may be something of an artistic snobbery element at work here, but that’s probably unfair; I think many people genuinely feel this sense of satisfaction in writing longhand.

I can understand this feeling but I don’t share it, perhaps because I associate handwriting with punishment. My handwriting was always a subject of criticism in my early school years, and later, minor infringement of rules was penalised by the requirement to write a number pages of script – always on blue paper, which had to be requested from one’s housemaster. This may well explain my dislike of writing by hand now. Anyway,there is good literary precedent for my view; two renowned authors, Henry James and Winston Churchill, used to dictate their books, although this is not something I’d want to do myself.

Whatever the reason, I find I write better and more fluently at the keyboard. At least for me, writing in this way probably does what writing longhand does for many others.

Qsf – a small fast accurate spam filter

There are plenty of spam filters available for OpenBSD and other BSD systems such as bogofilter or spambayes, but these tend to be overkill for a single user system like mine. Another possibility is bmf, which is described as aiming to be faster, smaller and more versatile than similar Bayesian filters. However, it doesn’t seem to have been updated since 2002 and I didn’t like it too much when I tried it out..

I prefer Quick Spam Filter (QSF), which I have used for a good many years – in fact, since before I changed from Linux to OpenBSD four years ago. Its last update was in 2015. I find that after training it is reliable, with only occasional false positives or negatives. It hasn’t been ported to OpenBSD but it compiles easily. It is available from its author.

The qsf documentation describes how to integrate it with procmail. I have a separate post with details of doing this with fdm – see the OpenBSD category or use the Search facility.

How to replace procmail with fdm

 

 

 

Procmail is still widely used although it is no longer maintained. Its former maintainer, on OpenBSD, Philip Gunther, has said that it should be removed from the ports list because of security vulnerability, although it is still there. A suggested alternative is fdm, which both fetches the mail from a POP3 server and filters it, thus cutting out the need for fetchmail.

I’d been using procmail with fetchmail for a long time but I thought perhaps I should switch to fdm. The main difficulty was that the documentation I found was less complete than what is available for procmail. Getting my mail filtering program to work was a particular problem.

My fdm.conf

I thought it might be worth posting my fdm.conf here for anyone who is seeking to make the same switch.

I use qsf as my spam filter but it should be easy to adapt the script to work with a different filter. It writes “SPAM” in the subject line of suspect emails. I’ve posted more details about qsf in a separate entry (see under OpenBSD in the Categories  list pr use the Search facility).


# fdm.conf -see the fdm manual for explanations of the code.

# Set the maximum size of mail.
set maximum-size 128M

# connection settings for my ISP’s mail server
account “my POP3 account” pop3 server “my POP3 server” user
“my email address” pass “my password”

# filtering rules (adapted from my .procmailrc
match “^(To|Cc):.*lyx-users@lists.lyx.org” action mbox “%h/Mail/lyx-users”
match “^(To|cc|Cc|Sender): .*misc@openbsd.org” action mbox “%h/Mail/openbsd”
match “^Subject:.*Sucuri Alert” action mbox “%h/Mail/sucuri”

# forward some mail to a different account
match “^From:.*Facebook” action pipe “%h/.forward”

# Spam filter rules
action “spamfilter” rewrite “/home/ac/bin/qsf -s”
match all action “spamfilter” continue
match “^Subject:.*SPAM” action mbox “%h/Mail/spam”

# Set spoolfile for incoming mail
action “inbox” mbox “/var/mail/ac”
match all action “inbox”


My .forward file:

smtp user@example.com