Crafted Mastering Emacs

October 18, 2022

Its been a while since I mentioned Crafted Emacs. The last update was back in July! There have been a few updates, a PR was submitted to update the straight.el configuration to use the newer version. There were a few other updates as well, mostly around some fixes needed in different places, and some additional documentation updates. I won't go into all of these on this post, instead I'd like to focus on the new module based on the book “Mastering Emacs”¹ by Mickey Petersen.

I bought this book way back in 2015 mostly to support the author and because, well, who doesn't need another book about Emacs, right!? The cool thing, this is more like a subscription because Mickey Petersen keeps updating the book with each new release of Emacs. Which is pretty cool, if you ask me. He also writes an article here and there to demystify common areas of Emacs and provide some example configuration as well. Between the book and the articles, there is a lot to learn and benefit from on the Mastering Emacs site.

A few months back, David Wilson and Mickey Petersen got together and worked out a cool deal that helps support the System Crafters YouTube channel. Basically, if you buy the book using this link, a portion of the proceeds go to the channel. And Mickey sells a book. And you get to benefit from a ton of really good information. Everybody wins!

It had been a while since I read the book. After the news just mentioned, I got inspired to read it again and put together a module for Crafted Emacs based on the book. The really cool thing the book highlights is the amount of built-in stuff immediately available for use. He goes over a number of configuration elements and covers the absolute basics every user needs to understand when using Emacs. Its like the built-in tutorial only better! The built-in tutorial does get you up to speed quickly in terms of basic operations, but the book expands on that information (repeating it in some ways) and does a great job of explaining how to go about configuring Emacs with very little actual Emacs Lisp knowledge and usage.

Of course, as this is a Crafted Emacs module, I will pretty much only use Emacs Lisp. I'm sure you aren't surprised. Not only did I use the book as inspiration, I also reviewed a few articles, notably the following:

Some highlights:

(customize-set-variable 'completion-cycle-threshold 3)
(customize-set-variable 'tab-always-indent 'complete)
(customize-set-variable 'completion-category-overrides
                        '((file (styles . (partial-completion)))))
(customize-set-variable 'completions-detailed t)
(if (version< emacs-version "28")
    (icomplete-mode 1)
  (fido-vertical-mode 1))

(when (version< emacs-version "28")
  (defun crafted-mastering-emacs-use-icomplete-vertical ()
    "Install and enable icomplete-vertical-mode for Emacs versions
less than 28."
    (interactive)
    (crafted-package-install-package 'icomplete-vertical)
    (icomplete-mode 1)
    (icomplete-vertical-mode 1)))

This bit uses the built-in completion settings. With Emacs 28, uses fido-vertical-mode which is in contrast to vertico as defined in the crafted-completion module. For users with Emacs versions before 28, provide a form to possibly install icomplete-vertical-mode if desired, otherwise just continue with icomplete-mode.

The completion-cycle-threshold value is set low. If there are 3 or fewer completions, hitting the TAB key will cycle through them, more than that and a *Completions* buffer pops up.

(add-to-list 'display-buffer-alist
             '("\\*Help\\*"
               (display-buffer-reuse-window display-buffer-pop-up-window)
               (inhibit-same-window . t)))

(add-to-list 'display-buffer-alist
             '("\\*Completions\\*"
               (display-buffer-reuse-window display-buffer-pop-up-window)
               (inhibit-same-window . t)
               (window-height . 10)))

(add-to-list 'display-buffer-alist
             '("^\\*Dictionary\\*"
               (display-buffer-in-side-window)
               (side . left)
               (window-width . 70)))

This bit configures some windows. For *Help* windows, keep using the same buffer, for *Completions* windows, make the max height 10 lines tall. This last one is to avoid having a lot of completions take over the entire screen. The former is to avoid the buffer proliferation that occurs when using the helpful package installed when using the crafted-ui module. The last bit of window configuration moves the definitions window to the left side of the frame.

(define-key global-map (kbd "M-#") #'dictionary-lookup-definition)

There is a keyboard binding provided to call the dictionary to get the definition of the word a the point as well.

I turn on winner-mode, also provided in the crafted-windows, but unlike that module, no additional keybindings are provided. The default keybindings are CTRL-LEFT or CTRL-RIGHT to undo or redo layout.

(with-eval-after-load 'ispell
  (when (executable-find ispell-program-name)
    (add-hook 'text-mode-hook #'flyspell-mode)
    (add-hook 'prog-mode-hook #'flyspell-prog-mode)))

Spell checking is turned on in different modes if there is a suitable spell check program, either ispell or, better, aspell. No keybindings are provided, but the default of CTRL-PERIOD might conflict with Embark if you have that installed and configured.

Finally, there are a couple of functions available to install dumb-jump and hydra. The dumb-jump package is useful for jump-to-definition situations, and hydra allows for creating a “menu” of keys to use so you don't have to remember every keybinding. Just use a single “leader” key and the menu will pop up. This is similar in concept to how the transient package works, which is used by the magit (and others) package.

A nice feature of this particular module is you can really simplify your Emacs configuration to nearly this:

(require 'crafted-defaults)
(require 'crafted-mastering-emacs)

and have a very reasonably complete Emacs configuration. Notable missing features are configurations for Org mode and any specific configurations for the various programming languages. Crafted Emacs has other modules which provide some of those features, but you'll need to fill in the blanks where functionality is missing.

Shameless Plug

If you enjoy configuring Emacs and tweak yours often, you might consider giving Crafted Emacs a try. I'm sure you'll find some rough edges, and when you do, I invite you to open an issue, or submit a pull request. Or, maybe you won't find any rough edges and this will be just the thing you need. I'm not holding my breath on that just yet.

If you enjoy crafting your computing experience, from hacking on Emacs, your Linux/Mac/Windows configuration, maybe your Guix or NixOS home configuration, or whatever it is… consider joining the SystemCrafters community!

Tags: #emacs

Footnotes

¹ See the Mastering Emacs website for more information.

From Rational to Crafted Emacs

July 16, 2022

Short update today, we renamed the Rational Emacs project to Crafted Emacs. Originally, the name was chosen because it sounded cool. However, after thinking about it, it seemed a bit elitist, as if this project was suggesting the only, well, rational way to configure Emacs, which was not the intent. There are many really good ways to configure Emacs, in fact the way you are configuring Emacs is one of them! Turn-key solutions like Prelude, Doom or Nano are also really good.

You might recall, in my last post I hinted this might happen and suggested a possible name: Crafted Emacs. This the new name. This name represents the project better in the following ways:

It doesn't sound elitist.
It fits better with the ideals of the System Crafters community.
It is closer to the intent, a good starting point to craft your own configuration.

During his stream today David Wilson started the process of merging the crafed-rename branch to the master branch. Unfortunately, there were some merge conflicts, so, while he went on with his stream, I managed the merge, resolving the conflicts and pushed the result to the master branch. David renamed the project in GitHub shortly after, and now the former project Rational Emacs is retired and the new project Crafted Emacs has taken its place!

If you happened to be using Rational Emacs, thank you for giving it a try! You may want to pull the changes to sync up with the new project name going forward. Git should work out just fine for you, but if you want to rename the remote to match the project name in GitHub, here are the manual steps:

From the command line:

git remote -v This will list the remotes so you can see the URL used. You'll want to copy that in a moment.
git remote rename origin rational
git remote add origin https://github.com/SystemCrafters/crafted-emacs or, if you happen to be using ssh: git remote add origin git@github.com:SystemCrafters/crafted-emacs
git pull origin master This just sycs the branches and tracking going forward. If you have already done a pull before, this should report everything is up-to-date.
git remote remove rational This optional step just removes the redundant remote from your list of git remotes for this project.

Update: 2022-07-16

The user Trout_Tickler on reddit helpfully pointed out this command to reset the git url: git remote set-url origin git@github.com:SystemCrafters/crafted-emacs which should replace all the above with a single command. Thanks!!

From Magit:

Press M for Remote
Press r to rename the remote, you will be prompted for the new name, type one (for example rational) and press enter.
Press M for Remote again, then press a to add a new remote. You will be prompted for a name, type origin and press enter. The url is presented for you, change it if you wish, or just leave it alone and press enter again.
Press M for Remote again, then press k to remove a remote, and choose the rational name (or whatever you named origin to earlier).
Press F to fetch updates, then either p or (my preference) u which should be listed as origin/crafted-emacs

To update your configuration, please see the section in the README which talks about it. I provided an Emacs script, you can run it with the following command:

emacs -Q --batch -l rational2crafted.el

And it should help. It makes a copy of your current config folder (ie ~/.config/rational-emacs/) and then updates the configuration from the new location. It doesn't work with chemacs2 configurations however. For those of you using chemacs2, you'll have to manually transition your configuration. That being said, read through the rational2crafted.el source, you can probably use some of the code from there to craft your own transition script. BLUF, it just renames rational to crafted everywhere. If you just want to do it manually, dired is your friend.

From Dired:

Open your crafted-emacs folder, maybe with this from the *scratch* buffer: (dired crafted-config-path)
Press %g and type “rational” to mark all the files having the word rational in them.
Press Q (yes, shift and the letter q) to replace rational with crafted (you'll have to type in those words), then repeatedly hit ! to change all occurences in each file.

There might be other ways to use dired to handle this situation, but that was the easiest way I could think of quickly.

So, there you go, brand new Crafted Emacs is now available for you to use to craft your own Emacs configuration!

Shameless Plug

Tags: #emacs

Rational Emacs Update

July 2, 2022

Another month gone, this update is rather shorter than the last one with only 3 closed pull requests and 4 new issues raised. Probably the most interesting thing from this month was the stream by David Wilson on the System Crafters YouTube channel where he spoke about the initial goals of the project and wrote his configuration based on Rational Emacs live. Here is a link to the stream beginning where he is talking about the goals and intentions for the project. To summarize his comments, Rational Emacs was intended to be a minimal set of configuration “blocks” to use when writing your own configuration. It might be enough to use without further configuration, but the expectation is some additional work would be needed to completely customize Emacs for your workflow. But, watch the video, he says it better than me.

What's coming next for Rational Emacs? Helper macros? More granular modules? Principles updates? David talks about this topic in his stream, including the potential for a new name! (Crafted Emacs anyone?)

Pull Requests

Customization groups were added to provide better organization when using the Customization UI from within Emacs. Customization options now appear in groups according to the module to which they belong.
A rational-lisp module was added to provide a base configuration for the Lisp family of languages: Common Lisp, Clojure, Scheme, and Racket.
A fix for the Doom modeline configuration.

New Issues

An issue was raised about where transient and other state files for Emacs should live. Currently, files like the bookmarks, and diary files live in the Emacs home directory rather than the Rational Emacs path. There is a PR in draft form to resolve this issue.
An issue was raised related to the friction caused by adding documentation when modules are added, specifically that the process is not quite straight forward. Some suggestions were made, especially to make use of GitHub actions, but no work has been made to configure those yet.
A discussion was started about not having the yasnippets package included and how that left the experience incomplete, especially as several packages provide snippets based on yasnippets.
A suggestion for naming modules was raised, specifically to follow a pattern of classifying modules by placing them into a hierarchy of folders, for example language based modules might go in the modules/languages folder. This was also a topic brought up on the recent stream by David Wilson.

Help Wanted!

There are still plenty of modules and documentation which need to be addressed. We'd love to have your pull requests for anything you see to make Rational Emacs more complete. Need ideas? Reach out to me or file an issue and we'll help you out!

Shameless Plug

If you enjoy configuring Emacs and tweak yours often, you might consider giving Rational Emacs a try. I'm sure you'll find some rough edges, and when you do, I invite you to open an issue, or submit a pull request. Or, maybe you won't find any rough edges and this will be just the thing you need. I'm not holding my breath on that just yet.

Tags: #emacs

Rational Emacs Update

June 8, 2022

Wow… Somehow I missed a month! Here is the update for June 2022!

I've tried to merge the pull requests as quickly as seemed reasonable while allowing the authors to update them at their own pace. In a few cases, it looked like the pull requests had been abandoned, so I pulled them, updated them, fixed any conflicts and merged them. As of the time of this writing, there are only two pull requests still outstanding, one of which I will be working on pulling, updating, and merging in the next few days or so. The other is a draft PR and I'm hoping the original author will come back to it and finish it as it looks like a really great addition.

Pull Requests

Some code cleanup and refactoring for the way the package repositories are updated occurred. Also several bits of documentation were added. Pressing C-h i and then finding the Rational Emacs menu entry now shows more information, in this case the modules rational-project and rational-completion.
Speaking of the rational-completion documentation, there are screenshots to help show the configuration working! I didn't even know that was possible (thought the texinfo stuff was text only!), so thanks to Stefan Thesing for the great work on that one!
The rational-completion module also received an update to include configuration for corfu, cape, and embark-consult.
New modules rational-ide and rational-python are now provided as some work toward addressing developers needs when using Emacs as a development environment.
A Troubleshooting section was added to the README which shows how to use packages from melpa by either installing them directly from the list-packages interface, or by using package.el's pinning feature. This information was provided in a few pull requests and Issues, but you had to dig around to find it. Since it has come up a few times, it made more sense to put it in a more obvious place to help users out.

Help Wanted!

Shameless Plug

Tags: #emacs

Rational Emacs Update

April 26, 2022

In my last blog post, I introduced the Rational Emacs project. Since then, I have become one of the maintainers for the project! I was very excited to be asked to join the team and am super thrilled to participate in this capacity. So, since I've been helping out a bit, I thought it might be nice to provide an update on what's been going on since my last post. That being said, I'm not going to cover every single pull request we have merged, but I'll mention some of the more impactful and major changes we've made.

And, as I've been looking into the pull requests as they come in, I've been using Rational Emacs more frequently to the point I've started adding the bits from my personal configuration I find myself missing. So, while in my last post I said I wouldn't be migrating my configuration, it seems that I've been doing exactly that! I'm nearly to the point of having my complete configuration ported over to Rational.

Discussions

Using package.el instead of straight.el and related topics saw a lot of traffic. In the end, the consensus was to move to using package.el as it is in alignment with the principles. There were a couple of posts lamenting the switch in one way or another, as those users preferred straight.el. I provided a minimal example configuration to show how to get back to using straight.el for those who preferred it. This was probably the biggest change as it affected every module and was a significant deviation from the initial startup scripts.
Using short answers received a number of comments. This setting controls whether Emacs will ask for a y or n response instead of yes or no. In Emacs 27, we use advice for the configuration as it is easy for a user to remove. Originally, we used fset but that is fairly impossible to undo, and as it was set in the rational initialization files, meant a user would have to modify a file that might get overwritten later with a git pull or possibly cause a local conflict which the user must resolve. It is easy to remove the advice, but that issue was raised again recently. In Emacs 28, there is a simple variable which can be toggled, no advice needed.

Pull Requests

Besides the aforementioned switch to using package.el and the use of advice for the short answers configuration in Emacs 27, there were these interesting pull requests:

The example configuration didn't work since we used doom-snazzy as an example theme and it is not provided by doom-themes. That was fixed by changing doom-snazzy to doom-one everywhere.
Loading of custom.el. This was also an interesting topic of discussion, mostly on the PR itself, but in the end we load the file by default. We prefer the use of customize-set-variable which allows constructors or setters attached to the defcustom declaration to run. This leaves the variables in a “pending” state as they are not written to the custom file, but are loaded each time Emacs starts. However, certain work flows specifically related to Org Agenda files and .dir-locals.el which write to the custom-file and would be “lost” on the next run of Emacs since it wasn't originally loaded at startup. There were several iterations on this and eventually we just settled for the simplest approach of just loading the file if it exists.
We started a group for Rational Emacs defcustom variables. There is still more to do on this to refine where module variables show up.
There is now a rational-speedbar module for those who want a tree of things similar to other editors. Speedbar is built-in to Emacs, launches in a separate frame, so it doesn't clobber your current frame's window layout, and has some nice features.
Info documentation has been started. There is plenty to do with this particular effort, but at least there is a start. If you try C-h i you'll find a menu option called Rational Emacs at the top off the list.

Shameless Plug

Finally…

I have really enjoyed working on this project! I'm super thrilled I was asked to participate as a maintainer, so I hope I'm doing a good job. I'll put another update post in the future, maybe this will be a monthly thing. It kinda depends on how much is going on with the project. If there is a lot, maybe a more frequent update pace makes sense. We'll see how it goes.

Tags: #emacs

Rational Emacs!

March 1, 2022

There is a new project in town! It is called Rational Emacs and is intended to be a sensible starting point for configuring Emacs. I've been contributing to it for a bit (along with many others, of course), and I have learned a few things along the way.

Project Principles

I'm particularly attracted to the principles behind this project. It's always good to start with some basic ideas around how to approach things:

Minimal, modular configuration. The intent with this is to provide modules that can be independently included and provides as little as possible for configuration.
Prioritize built-in Emacs functionality. I love this! The idea is to use very few external packages, and the ones to use should be chosen for their enhancement of core features rather than complete rewrites. The examples are:
- project.el instead of Projectile
- tab-bar-mode instead of Perspective.el or others
- eglot instead of lsp as it is built to leverage built-in functionality.
- corfu instead of company
- And the list goes on.
Works well in the terminal. As some people prefer the terminal, or have no option (for example, over an ssh connection), this should still work well.
Helps you learn Emacs Lisp. This is a great feature! Things should (ideally) be consistently and clearly written so it is possible to understand what is going on in the configuration.

For myself, I really resonate with these ideas. My own configuration has been growing more and more towards these principles for a while. I stopped using use-package because I wanted to figure out how to manage packages and configuration without macros hiding that from me. I still am using a lot more packages that are not quite that close to core Emacs, but that will be changing soon.

For the things I've contributed, I've tried to keep things reversible as well. So, whatever configuration I provide, the eventual user should be able to easily reverse or override that decision with one of their own.

The Journey So Far

There was an initial flurry of activity the first week and a lot got done quickly. I think for the most part, the community of people participating have agreed on just about everything. The discussions have been super interesting, insightful, and thought provoking. There has been a lot to gain from just reading people's posts on issues and pull requests. It's a very rich field full of diversity!

The second and third weeks have seen much less activity. Life happens, and this project isn't the highest priority for the maintainer. Inevitably, things have slowed to a temporary halt. I'm sure additional people will eventually be added to list of maintainers, but for the moment, we are a little bit stalled.

Stuff I've Learned

There is so much, I'm not sure I'll capture it all here.

Skeleton and Tempo are interesting replacements for the venerable Yasnippets package, and both are built-in. Neither are documented very well (at all!) but reading the code was interesting and then providing a skeleton for “rational modules” was fun.
Tempel is a nice enhancement to tempo.
Corfu as a Company replacement is quite nice. Sometimes I like the way company automatically pops up a list of completions, and sometimes I don't. With corfu, I have to remember to press the TAB key to get the menu, which, in most cases, I do like. I used a different key with company, but TAB works for me.
Abbrev and dabbrev are still relevant. I have used these in the past, but found yasnippets to be more approachable and functional. Add tempo/tempel and suddenly these come back to life. Who knew?

Still learning my way around with these (and other) tools already built-in to Emacs, but I don't see that I'm any less productive using them and it didn't take any time (for me) to embrace them fully.

Straight vs Package

I think one of the fun things about this project is I don't quite agree with one of the decisions. I don't think it is a bad decision, I just think it is one of the corner cases that doesn't quite line up with the principles. That decision is the one to use straight.el rather than package.el. The latter is built-in to Emacs and the former is not. Today, I tried to see what it would take to override that decision in my personal configuration. Turns out, it wasn't so difficult as to be impossible. My configuration of package.el takes advantage of a lot of things that aren't well known; for example elpa priorities and pinned packages. The hard part, though, has to do with performance. I did manage to get it to load Emacs a little faster than using straight.el, but only barely. The key was not calling package-refresh-contents on startup¹.

Using straight.el is a big win in a couple places:

No automatic package loading and activation; it only loads the packages you choose to configure. So, you could have something installed you aren't using and straight won't load that package. There is a way to do something similar with package and I leverage that configuration, but it's not the default as it is with straight.
No need to call package-refresh-contents to download a list of packages. This is done when you install straight and it gets updated intelligently. This is a bit more of a conundrum with package.el as you need to call package-refresh-contents at least once before you can actually install packages, especially since we are adding more than just the “gnu” elpa to the list of sources from which to get packages. You also don't want to call that function everytime Emacs loads. There is a way to handle it, but it does complicate things a bit and straight.el is, well, pretty straight forward.
You can freeze versions and make your configuration reproducible with exactly the same package code on multiple machines. That is harder to do with package.el but possible. With straight, it's just a call to straight-freeze-versions.

There is a lot more to it than all that which makes straight a cool tool to use.

On the cons side of things though, there are at least these:

No interface to see what you have installed. Using list-packages, you won't see any of the packages you installed with straight. You can sorta figure it out using dired on the user-emacs-directory/straight/build path, but you won't see what versions are installed or if anything needs to be updated.
It is not built-in, which seems to be in conflict with the aforementioned Project Principles.

While it seems like I have gone on a rant about it, I do indeed like straight.el and I don't think its use in this project is wrong. We can make the choice to use it as the benefits outweigh the costs, at least for a given target audience.

Target Audience

I'm not exactly sure who the target audience is. You could say, it is anyone who wants to use Emacs. I'm not sure that would be accurate though. Any new user coming to Emacs may not want to use Rational Emacs as a first attempt at configuring their Emacs. In theory, they could… just need to create a config.el file and use require statements to pull in the modules they want and they get a rational set of minimal defaults out of the box (see the example-config.el from the project repo as an example). Much more than that though… like adding additional packages or (re)configuring some of the packages we install will require some Emacs Lisp knowledge. That, to me, makes it seem like we are targeting someone who has used Emacs for long enough to not be scared away by having to type lisp as the configuration language. Do we call that an intermediate beginner? How much Emacs knowledge are we expecting they have to use this project? We do not intend to be a Doom or Spacemacs replacement. We are closer to Prelude philosophically.

Where From Here?

We'll continue to develop Rational Emacs. Is it usable today? Sure it is! At least with the caveat you are willing to read through the code to see what it does and possibly write (some of) your own configuration.

Will I migrate my configuration to Rational Emacs? Probably not, but we'll see. Will I take some of the ideas from there and implement them in my configuration? Absolutely. Already doing it. All that being said, I am working on trying to see if migrating my configuration to Rational Emacs can be done or if I just end up rewriting and overriding everything from Rational Emacs. If it is the latter case, then I'm not really using Rational Emacs, which is kinda where I am at the moment.

I am enjoying the project quite a bit, as well as the evolving community working on it, so I do plan to continue submitting pull requests and being active on the discussions on the issues list.

Tags: #emacs

Footnotes

¹ For the curious, I created two helper functions, one to check to see if every package in a list was installed and a second to map over that list and install them. Then I have an unless block check the first and call the second. If the first call passes, then the second function (and thus package-refresh-contents) is never called.

Emacs Package Versions and MELPA Stable

November 4, 2021

This is a bit of a rant. Hopefully, thought provoking, and absolutely not blaming anyone for anything. BLUF: I value a released version of Emacs packages, I choose to use MELPA Stable hoping the releases there are meaningful and reasonably stable in the sense that they don't change with every single commit, that they represent a point in time where the developer felt like putting a mark on the wall that certain features were “done enough” that users could upgrade (after all, its just a git tag command, so why not?).

A while back I rewrote my Emacs configuration¹ with certain goals in mind. I won't rehash all of them, but one of them was to use only versioned packages as much as possible. That particular goal is still a fairly important aspect of my configuration. While I am a developer, I am not an Emacs contributor (yet.. maybe), so I use Emacs much like most of my tools. In other words, it's a tool. A fun, do everything, hacky kind of tool, but a tool to use to get things done. I want stable packages so my tool doesn't break and I waste time trying to figure out why it is broken and how to fix it.

Reading the GitHub page for the MELPA² site, there is a section on MELPA-Stable. It has the following text near the bottom of that section, “Note that the MELPA maintainers do not use MELPA Stable themselves, and do not particularly recommend its use”. There are even several posts in different places³,⁴ which suggest not using MELPA Stable for different reasons: missing packages, lack of quality, lack of coordination between packages, etc.

Yet, I get messages from my installed version of lsp-java 3.1 which gives a warning indicating dash-functional is deprecated and please use dash-2.18! It actually recommends a version number! If you happen to be using MELPA, you'll likely never see this warning, and if you do, it's useless since the version number from MELPA is a timestamp. Unfortunately, there has not been a recent release of lsp-java which fixes the issue, so I'm still getting the warning. Seems to not hurt anything, but the solution is to stop using a released version lsp-java and instead switch to a development version.

Another user on reddit ran into a problem⁵ using a version from MELPA which broke his org agenda. He resolved the problem by providing a do-nothing defun. Comments indicated the code should have been guarded with conditionals and checks and things, but wasn't. These are the kinds of issues I'd like to avoid by using released versions.

Unfortunately, just tagging a package with a version number to get it to build in MELPA Stable provides no additional guarantees about quality or completeness. Which is one of the arguments to why you shouldn't use MELPA Stable. I'm not sure I completely agree with this, I believe the package maintainers are doing their best to provide quality software as best they can, and, at least in theory, they have performed a bit of testing to make sure their package works before they push their code. They are human and things get missed, mistakes happen etc, but why not release something periodically?

Consider this, if I add a dependency to my pom.xml for my Java project, I use a released version. If I install a package for my distribution of Linux⁶ I get a released version. I'm going to go out on a limb here and suggest it is at least uncommon, in general, for anyone to want a development version of a piece of software. Some absolutely do, and they should have the freedom to do so. However, that is exactly what MELPA users get! A development release, essentially whatever the developer checked in last when the MELPA build daemon ran. That isn't necessarily bad if you understand and desire the most bleeding edge package updates. If you don't though, then what?

Ideally, MELPA Stable would meet the following expectations:

Slower updates, no expectation of a version update just because something was committed to master. Sometimes commits come in groups of changes around single concepts. But a new release when something is reasonably complete would be appreciated.
Some amount of testing occurred to ensure the version is not likely to break something. Unit tests for the win, right! If unit tests are being written and they all pass after a change, why not make a release?
New releases as dependencies change.

Most of this can be automated with tooling, ala GitHub Actions and similar, so the burden on the developer writing that amazing package, ideally, would not be high. Some initial investment, certainly, but long term those things which can be automated do help.

It would also help if Emacs' package.el were updated to support multiple versions among other things. There is currently no way to go back in time and choose to use a different version of an Emacs package using built-in tools. Indeed, there are a number of ToDo items listed in package.el and that list is likely not exhaustive.

There is a lot of technical work to improve the state of Emacs packages, which is significantly better today than it was a few years ago before we had package.el. Also, there are third party solutions like straight⁷ and others which solve the problems of versioning different ways. Personally, I hope the Emacs community can embrace MELPA Stable, or similar repositories (elpa.nongnu.org?) and give people like me a more reasonable choice to use released versions instead of development versions.

Tags: #emacs

Footnotes

¹ https://jeffbowman.writeas.com/rewriting-my-emacs-config

² https://github.com/melpa/melpa

³ https://www.reddit.com/r/emacs/comments/etikbz/speaking_as_a_package_maintainer_please_do_not/

⁴ http://blog.jorgenschaefer.de/2014/06/the-sorry-state-of-emacs-lisp-package.html

⁵ https://www.reddit.com/r/emacs/comments/qil2qh/symbols_function_definition_is_void/

⁶ https:://archlinux.org

⁷ https://github.com/raxod502/straight.el

Writing Documents with Org Mode

July 24, 2021

Motivation

For work, I often find myself writing documents for the projects I'm on. Usually, I just open Emacs and start a LaTeX file, but this time I thought I'd try using Org mode for writing the document. There were a couple of motivations for this, in one case, I needed to generate a presentation and I'd seen enough posts about exporting to a PDF via Beamer to finally take the plunge and try it for myself. In the second case, I had been taking a bunch of notes and computing metrics in tables.

Beamer

Getting started with this document took a little longer because I didn't know what I was looking for. I needed to add #+STARTUP: Beamer to the top of my file and then press C-c C-c on that line to get the Beamer export options. The other confusing thing about exporting a presentation with Beamer is how the headlines work. Most of the examples I saw used #+OPTIONS: H:2 or #+OPTIONS: H:3. This sets the header level to export.

#+options: H:2 * header 1 This header is not exported ** header 2 This header is exported as the title of a frame *** header 3 This header is exported as a subsection on a frame

Once I figured out how that structure worked, I made quick progress on my presentation.

The themes available look nice, however, for a lot of them, I couldn't tell a difference between one and the next. I looked at the Beamer theme matrix, each row represents a theme, while each column represents a different color scheme. Looking across one row, several of the color schemes look identical to me. I hope I'm not missing something fundamental, but there doesn't seem to be a lot of variation. What is provided does look good. I chose to use the Dresden theme with the lily color scheme.

PDF Documents

This one was more straight forward. Out of the box, the result is pretty good. There are some tweaks to be made. Adding things to the preamble is as simple as adding another #+latex_header: option. In my case, I added the following:

#+latexheader: \usepackage{array} #+latexheader: \setlength{\parskip}{1em} #+latex_header: \newcommand{\tabcap}[2]{\caption{\label{tab:#1}{\textsc{\scriptsize #2}}}}

I have several tables in this particular document and I wanted to change the default font and size for the caption, so I provided a new command for that purpose. The array package is also used for the tables to help with column layout. I want a blank line between paragraphs, so I change the default parskip value.

Most of the document is based around the data I present in the metrics tables, so I wanted to control a couple aspects of the layout. Each table has a #+NAME:, a #+CAPTION: and some layout options specified with #+ATTR_LATEX:. The only challenge I did not overcome was the placement of the table caption. On export, the caption appears above the table, but I would prefer it to be below the table. I did not figure out how to adjust the placement of the caption, so I manually modified the tex. The downside of that approach is now I have to remember all the manual changes I make if I choose to maintain the org file and export again. In this case, I just moved to modifying the tex file instead of the org file.

Conclusion

I frequently use Org mode for managing my day at work. I track what tasks I'm working on, meetings to attend, time spent in each, and general note taking for each. It is nice to be able to section parts of the file into logical sections and be able to quicly navigate between them. Using the same power when writing documents to distribute as PDF files or as presentations is just as rewarding. You can even setup the org file for different kinds of exports by using C-c C-e # and choosing a template of options to insert. In the case of a Beamer file, remember to add the appropriate startup option and evaluate it first so the template exists to insert. In general, Org mode is a great place to start writing documents, or even finish them if you don't need to modify them manually in the external format.

Tags: #emacs #org #pdf #beamer

Rewriting my Emacs config... again!

June 9, 2021

Motivation

Recently, I looked into other package management tools for Emacs. I looked at ElPaso, Borg, and Straight. They all seem to be ways to avoid using the Emacs built-in package.el, which I'm not all that interested in doing. I find the simplicity of being able to list the available packages I might be interested in installing too useful to switch away to something else. They are interesting projects, and you should probably look into one or more of them to see if they make sense for your use.

Even though I was already using use-package, and am generally happy with how it works, I thought it might be fun to re-organize and rewrite my configuration. There is a lot of cruft and I needed to sort that out anyway.

For goals, I chose this list:

Rewrite without use-package.
Keep start-up time to a minimum (my startup time was just above 2 seconds).
Programmatic activation of chosen modules with byte compilation.
Prefer versioned packages over “latest commit” packages (ie prefer other repositories versions over MELPA versions).

Influence

As mentioned previously, I used Prelude for a while and still take inspiration from its code and layout.

As a summary, Prelude is organized around some core configuration with some modules which can be configured or mixed in automatically depending on the mode needed for editing a file. For example, editing a clojure file would automatically activate the Clojure module in Prelude. This is a cool feature, but I'm not that ambitious, so it is not one I tried to emulate in my configuration. However, how the core and modules are organized, I did take a bit of inspiration from that.

Approach

Following Prelude's lead, I have the following directories:

core: This is where the base configuration exists. It contains the elisp needed to install, activate and configure the base set of packages I always want to be enabled.
modules: This is the list of “modules” for various programming modes or other tools which I want to use, but not necessarily have activated on startup. For example, I might be interested in editing the Lua configuration for my AwesomeWM configuration, but since I do that rarely, I don't want it activated unless I edit a Lua file. This directory is not on the load-path. More information below.
personal: This is where personal configuration goes. Most of the configuration will reside in either the mycustom.el file which is for general customization of various modules or for things that just don't fit anywhere else. The mymodules.el file is where I activate the modules I want to use. This directory is on the load-path, modules are symlinked here (or when I am on a Microsoft Windows machine, copied here).
sample: This folder contains an example mymodules file to copy into the personal directory and edit there.

I make aggressive use of autoload and with-eval-after-load to control how and when packages are enabled. There is still more to do, I think using mode hooks would be a cleaner approach for some things, so I'm still considering how I might implement that concept, especially for programming modes and LSP.

Packages

This is how I configure my package-archives, I have it listed in the order I prefer to get packages, but the ordering here is not relevant.

(setq package-archives
   (quote
    (("gnu"          . "https://elpa.gnu.org/packages/")
     ("nongnu"       . "https://elpa.nongnu.org/nongnu/")
     ("melpa-stable" . "https://stable.melpa.org/packages/")
     ("melpa"        . "https://melpa.org/packages/")
     ;; [2021-03-31 Wed] org currently at 9.4.5, but this is about to
     ;; go away after the release of org 9.5. Org will be distributed
     ;; through GNU elpa and org-contrib will be distributed through
     ;; nongnu elpa (see above for both).
     ("org"    . "https://orgmode.org/elpa/"))))

I continue to use the orgmode.org/elpa until the release of Org 9.5, and will remove it from the list at that time.

To use melpa and still prefer versions, I use a feature of package.el called priorities. If a package exists in a higher priority elpa, then install from there, otherwise, try the next lower priority elpa and so on until the package is found. Some packages are only found in melpa so I get them from there. This allows me to get a versioned package from one of four other elpas before trying melpa

N.B. elpa in the last paragraph refers to the Emacs Lisp Package Archive specification rather than a specific repository.

This is how I configure the priorities for packages, higher numbers are preferred over lower numbers when retrieving packages:

(setq package-archive-priorities
      '(("gnu"          . 99)
        ("nongnu"       . 80)
        ("org"          . 75)           ; see comment above, this will
                                        ; be going away
        ("melpa-stable" . 70)
        ("melpa"        . 0)))

I don't want all of the packages I have installed activated by default, so I pass t to package-initialize to not activate anything and allow the rest of my configuration to do that work.

(if (file-exists-p package-user-dir)
    ;; don't activate all packages, instead they will be added via
    ;; mypackage-require, which will install if needed as well.
    (package-initialize t)
  ;; on a new system, need to activate everything initially
  (package-initialize)
  (package-refresh-contents))

Finally, Prelude has a nifty idea of tracking a default set of packages that should be installed initially, and to which additional packages are added as they are activated later. I cloned that idea directly from Prelude and use it as well. I do attribute the code and give credit to the author in my code. I did change some of the names to match the rest of my code (ie mypackage- instead of prelude-).

Modules

The modules portion of my config contains specific configuration files for each of the programming, text, and utility (ala docker) modes I currently use. The mymodules-config.el has functions that create symlinks in the personal directory and then byte compiles the file there. There are two variables used which can be configured using setq:

mymodule-available-dir: This is the directory where the modules live. They are symlinked or copied from here.
mymodule-active-dir: This is the directory where the modules are symlinked (or copied) to make them active.

To create the symlinks, I thought a macro would make sense:

(defmacro mymodule-activate (module)
  "Install symlink to source and compiled MODULE files

Gets MODULE from MODULE-AVAILABLE-DIR to create a symlink in
MODULE-ACTIVE-DIR, byte compiles the module after creating the
symlink and then loads the MODULE. If the MODULE is already
installed, just load it."

  ;; initially create names for use in the macro itself. This is not
  ;; really needed as the only input is MODULE which is a string, but
  ;; it means the symbols created can *only* be used here in this
  ;; macro.
  (let ((module-name (make-symbol (file-name-base module)))
        (load-name (make-symbol "load-name"))
        (available-module-file (make-symbol "available-module-file"))
        (active-module-file (make-symbol "active-module-file")))
    `(let* ((,module-name ,module)
            (,load-name (file-name-base ,module))
            (,available-module-file (expand-file-name ,module-name
                                                      mymodule-available-dir))
            (,active-module-file (expand-file-name ,module-name
                                                   mymodule-active-dir)))
       (if (file-exists-p ,active-module-file)
              (load ,load-name t)
            (if (string= "windows-nt" system-type)
                ;; if this is Microsoft Windows, we can't use symlinks, so
                ;; copy the file instead.
                (copy-file ,available-module-file mymodule-active-dir)
              ;; Linux or Apple Mac can handle symlinks
              (make-symbolic-link ,available-module-file
                                  mymodule-active-dir
                                  t))
            (byte-compile-file ,active-module-file)
            (load ,load-name t)))))

However, since symlinks don't work in Microsoft Windows, I have to check for that and just copy the files from the available directory to the active directory.

Customization

Out of the box, I don't need to do anything. If I don't have a mymodules.el file, then just the default packages are loaded and I can still get a lot of work done. Things like Org-mode, Magit, Ivy, etc. allow me to do enough from day-to-day. I tend to spend a lot of time in Org-mode tracking work, tasks, meetings, notes, etc. so the base is sufficient.

However, as I also need to edit Java, Json, XML, and other things like Docker files and diagrams, so I activate those modules on the machine where I do that work. This gives me the flexibility to install the modules I use to configure Emacs for the machine I'm on at the time. My work configuration contains different modules than the configuration on the client environment, and from my personal computer.

Conclusion

This has been an interesting exercise! It took about eight hours to accomplish the rewrite to the point I have it now. I did clone some code from Prelude, and I did reach out to Bozhidar for permission, which he graciously provided.

There are still some things I think could be improved in this configuration. For example, I need to look into using mode hooks more frequently. Some of the autoload code does not activate the way I think it should, but a mode hook would most likely fix that problem.

As for meeting the goals:

I have replaced use-package. To do so, I used pp-macroexpand-expression to understand what was happening and then implemented a solution using either or both of autoload and with-eval-after-load.
Startup time is about 1.6 seconds.
The macro module-activate handles making symlinks or copies, byte compiling the module and loading it.
Using package priorities, I get versioned packages, or a version from MELPA if that is the only repository where a package is distributed.

The code: git repo.

Tags: #emacs

The value of Emacs pre-made configurations

May 15, 2021

I was watching this video on the System Crafter's YouTube channel during which he considers whether or not using a pre-made Emacs config is a good thing. He has several good points on both the pros and cons sides of things. I won't mention all of them, you should go watch the video for that, but here are a few things that resonated with me.

Pros

Makes getting started easier. I know from experience, configuring Emacs to do cool stuff can be a bit of a daunting challenge when first starting. I've been using Emacs for a number of years, hand-coding my configuration and trying to get it to do the cool things I saw others doing on various videos. Now it is something I enjoy a bit more, but I eventually ditched my hand-coded configuration and went with a pre-built config for a while (more on this in a moment). Things worked out of the box in a nice clean way. So, even though I “started” kind of in the middle of my experience, this reasoning makes sense to me.
Better integration between different packages. If there is one thing I have struggled with and have seen in posts on reddit and stackoverflow, its the many ways to do really common things, like completion (ac or company?), fuzzy stuff (ivy, helm, consult, selectrum, counsel…??), language tools for software development (slime, sly, lsp… ???). A lot of the pre-built configurations just make decisions on these and then make them work.
Often there are more learning resources. Yes. In the cases I've reviewed (and by reviewed, I mean the ones whose documentation I read when I was deciding which one might be most to my way of thinking and working), there has been fairly good to amazingly good documentation. The approach is usually how to work with the configuration to make the smallest changes needed to make it “yours” and leave the rest alone. Each claims to provide sane defaults, which I find largely to be true for some definition of sane.

Cons

It might convince you that vanilla Emacs is too hard to use. The idea here lines up with the next one. Basically, if you look at the code base for most of the pre-made configurations, they are not small. So, it may lead you to believe this is the minimal viable source needed to actually configure Emacs to be usable. This is not true. A good friend of mine has been using Emacs for a number of years and his configuration is very short. He doesn't configure a lot of the whiz-bang features so popular in other IDEs, editors or even within Emacs itself. Works fine for him. Emacs, without any configuration or additional packages is still a very powerful tool. It just depends on what you are used to and what your expectations are. The more “things” you expect Emacs to do, the more you need to do to configure Emacs to work that way.
Specialized configurations can make things more confusing. One of the arguments both for and against using a pre-made configuration is that it gives you a way to see how someone else put things together so you can learn from it and then, eventually, write your own. But trying to read the code for some of the pre-made configurations can be just like those tween choose-your-own adventure books from the 1980's. Trying to understand why something works in that configuration but not so much when you try to write it in your own custom configuration can be confusing. It definitely requires you to learn a lot, but can lead to enough frustration to rage-quit from time to time. Additionally, you need things like a pre-made configuration provides to easily navigate through a call stack in source code when you are reading it.

My journey

I started using Emacs a long time ago, I was fascinated by the Lisp family of languages and thought using Emacs would help me learn and use that particular language. While that is absolutely true, Emacs Lisp is just another dialect and knowing it is not the same as knowing Common Lisp or Scheme. But I digress.

I was also very interested in how productive people were when using Emacs and I wanted to that for myself as well. I did learn enough Emacs Lisp to sort-of make things kinda work and I was generally happy with most of my work flow being in Emacs and jumping out to other tools for more hard-core editing. I just kept coming back to Emacs because I found myself more comfortable here.

I started with Prelude

I really wanted some of the features I'd seen in other IDEs and editors, and so I researched a few of the pre-built configurations and settled on prelude by the venerable Bozhidar Batsov. It should be noted, at the time I was looking, there weren't that many pre-built configurations. Spacemacs was around, I think Doom Emacs was just getting traction, the Emacs Starter Kit existed… those are the ones I recall reviewing. I didn't do an exhaustive search, so there might have been others. I chose prelude because I wanted something as close to vanilla Emacs as possible and something whose code I could use as an example to learn from. I didn't want Vi in Emacs, I can use Vi if I really want that experience, so nothing against Vi, just not what I wanted to use. I didn't want a bunch of hijacking macros around things like package.el. And I didn't want a literate configuration. Org mode is super cool, and I definitely find it to be one of the most useful packages I use, but I wanted my configuration to be straight code. I can write comments in my code just as well as writing text around code snippets, and with hide-show-mode I can collapse things if I want to. I wanted to be able to use the existing features of Emacs and maybe a few well integrated packages. I got that from prelude.

And then I wrote my own

After using it for a while, and learning from how it was written, I wrote my own configuration. I setup how my configuration was organized in much the same way that prelude did it, although without quite the same level of sophistication. My only complaint was the 19 seconds it took to startup. That was something I just thought I'd have to live with in the post-prelude world.

And then I did it again

I started seeing a lot of people moving their configuration to use-package most often citing the simplicity of configuration and the speed at which Emacs was starting. So, I took the plunge and rewrote my entire configuration using use-package to install, load and configure the packages I was using. For fun, I went back to using prelude, reconfigured a few things, and got my start time to around 4 seconds which compares favorably with my current configuration that starts in just over 2 seconds. Slow by some people's standard, but I start it in daemon mode and the use emacsclient so I don't keep starting/restarting Emacs, and starting emacsclient is nearly instantaneous. Still, I think there are a few things I could tweak a bit to get the prelude version to start faster and shave off a couple (or more) of those seconds.

So those pre-made configurations have value

From my experience, I learned a lot from starting over with my configuration by using prelude and then growing back into a personal configuration heavily inspired by how prelude is written. I have also looked at the source for Spacemacs and Doom Emacs and even Emacs Starter Kit and there is a lot to learn from those as well. I think the “pros” they provide out weigh the “cons” enough to recommend them. I would suggest identifying what you would want from Emacs, then reviewing the goals and source code for several pre-built configurations before just jumping on one and being frustrated that it doesn't do feature x in some way you think it should. Try looking on the Emacs Wiki for a list of pre-made configurations that might make sense to you.

And Bozhidar, if you happen to come across this page, thank you very much for your work on Prelude. I've learned a lot.

Tags: #emacs