Dotfiles Management - mitxela.com forum

Welcome. Please log in or register.

Dotfiles Management

arti	Posted: 22 Dec 2022, 07:47 PM
Member Posts: 1 Joined: 22-December 22	I also manage my dotfiles in a similar way. One thing I do differently is that I can enter/exit into an special environment where normal git commands and other programs can use the dotfiles repo without extra config function _config_activate { export GIT_DIR=$HOME/.cfg/ export GIT_WORK_TREE=$HOME CONFIG_ENV="config" # used to show alternate shell prompt } function _config_deactivate { unset GIT_DIR GIT_WORK_TREE CONFIG_ENV } # Dotfiles config entrypoint function config { # shortcut for running git commands directly if [[ -n "$1" ]]; then _config_activate git $@ _config_deactivate return # Handle entering and exiting dotfiles environment elif [[ -z "${CONFIG_ENV}" ]]; then _config_activate else _config_deactivate fi } This setup allows me to have a nice shorthand for quick commands like config status config add ... Or I can enter dotfiles environment and just use any git command with fully working tab completion. config # enter env git status # works on dotfiles gitk config # exit env All of my computers use the same dotfiles repo. So far that has worked out quite well. https://github.com/artizirk/dotfiles Your tracked files summary function is pretty neat. Definitely going to borrow that! -------------
	[top]

mit	Posted: 23 Dec 2022, 10:51 AM
yeah whatever Admin Posts: 566 Joined: 4-May 16	Oh that's a neat idea, that does seem more versatile than sticking it all in the aliases. By tracking the OS files as well I've kind of forced myself to have a different branch for different machines. I may re-think the way I do it later. Yes I'm quite pleased with the summary. One tiny change I made to make it copy-pastable is to use \e for the colour changes with echo -e. In the original script it's just echo and literal escape characters (by pressing ctrl+V, escape). I realised as I made this change that if ever the commit messages have escape sequences in them it might not display properly. I guess worrying about this can probably be filed under 'yak shaving'. -------------
	[top]

shreeve	Posted: 10 Jan 2023, 04:52 AM
Member Posts: 4 Joined: 10-January 23	Oh, man! This post got me spun up for a while today. The issue is that I was trying to recreate the nice little commit synopsis, but it is actually really hard to get the status, filename, date, and last commit message in an efficient way. Like, sort of crazy hard to do without being inefficient (ie - calling git commands multiple times for each file). In the end, I sort of had to have a hybrid approach. Here's what I ended up with: In my ~/.zshrc file: # fs: full-send is a filesystem wide git repo; fs init -b macos ~ && fs config --local status.showUntrackedFiles no alias fs='(){( export GIT_DIR="${HOME}/.git-fs" GIT_WORK_TREE="/"; cd /; [[ $# > 0 ]] && git "$@" \|\| git grok )}' Essentially, I create an alias called `fs` for meaning it's a filesystem-wide git repo. Using "f" and "s", I just called the project "full-send", like we're going full monty on putting the entire filesystem (minus untracked files) in a git repo. Notice that, when any parameters are passed, all this really does is spawn a subshell, change the directory, set some environment variables, and the just call regular git with those parameters. However, when no parameters are called, this will run `git grok`. Of course, there is no such command as `git grok`. But, git is pretty smart and it will look in the path for a command named `git-grok`. Essentially, when you call `fs` (full-send) without any arguments, it will just set the filesystem wide environment variables, cd to '/', and call `git-grok`. So, you can customize `git-grok` to do whatever you'd like. Using your example, which I realize was actually very clever and compact, I created something similar but perhaps with a little more control of what's happening. Here's the script for `git-grok`, which I placed in my path. #!/bin/bash # emit ANSI color codes if allowed, otherwise they are ignored •() { IFS=";" printf "\e[%sm" "${@:-0}"; }; [[ -t 1 ]] \|\| •() { :; } [[ "$(pwd)" == "/" ]] && root="/" \|\| root="" IFS=$'\n'; for line in $(sort -ufk 1.4 <(git status -suno .) <(git ls-files \| sed 's/^/ /')); do stat=${line:0:2} file=${line:3} info=$(git log -1 --date=format:"%Y-%m-%d %H:%M:%S" --format="%cd•%s" -- "${file}") case $stat in # NOTE: codes should be TWO digits so columns line up! "A ") tint=$(• 32) ;; # added is green " D") tint=$(• 31) ;; # deleted is red " M") tint=$(• 33) ;; # modified is yellow " ") tint=$(• 37) ;; # unchanged is white ) tint=$(• 36) # others are cyan esac echo "${tint}${stat}•${root}${file}•${info}$(•)" done \| column -t -s • So, if I want to do any normal management of this file-system wide git repo, I just use `fs ...` (for whatever git command I want to use). If I want to see the status of the files in the repo, I just type `fs` by itself and get a very nice list. A nice twist is that I can now use the same `git-grok` script with all my other git repos. I just run `git grok` from any of my normal repos and they can use the same nice display. I just wanted to post here and let you know I was very* impressed with your idea and work on this and also to log my insanity here, so at least it can be documented somewhere and about four Mountain Dews worth of work won't have gone completely to waste! Haha... -------------
	[top]

shreeve	Posted: 10 Jan 2023, 05:03 AM
Member Posts: 4 Joined: 10-January 23	-------------
	[top]

mit	Posted: 11 Jan 2023, 04:23 PM
yeah whatever Admin Posts: 566 Joined: 4-May 16	Thanks for sharing. I'm kind of amazed by the variety of responses I've had to the post about dotfiles, seems there are an awful lot of ways of managing things and lots of people have their own personal way of doing it. Your git-grok function is another nice way of doing it. I think the reason git doesn't have exactly this built in is because in most normal repos, you can just do ls, but that's not an option for dotfiles, especially if you're using "/" as the work dir. On the other hand, it is quite pleasing to have this kind of output, which is probably why github and so on do it. In terms of the colours, I do like how the whole row has been highlighted, however I'd be tempted to keep green/red as staged/unstaged changes, I guess just because I'm used to those associations. Then again for actual dotfiles, once they're tracked I generally only use commit -a so staging is kind of irrelevant. -------------
	[top]

shreeve	Posted: 15 Jan 2023, 10:51 AM
Member Posts: 4 Joined: 10-January 23	Thanks for your reply and thoughts. My last code tried to reduce the number of program executions to determine the current status of the repo. It was significantly faster than without, but even so, it took nearly 45 minutes to show the results for the Ruby github repo! I ended up with a way to get the list of files extremely quickly (git ls-files) and the list of modifications extremely quickly (git status -suno .), but the issue was trying to get the log messages for every file in the repo. Turns out that it'll take forever on very large repos. But, there must be another way. Instead, I ended up getting the entire log history of the repo (every commit and every file). Using this, we can quickly identify the last time each file was altered. By using the git hash to quickly look this up, we can get a massive improvement in efficiency and time reduction. Using this approach, generating the commit history for the Ruby repository went from 46 minutes down to 4.6 seconds, which is a 600x improvement. The flow is now that the `git-grok` program will call my `grit.rb` Ruby code, who does the hard work now and then spits it back to the `git-grok` shell script who then takes the output and puts it into columns using the standard `column` command line tool. Here's the update `git-grok` code: #!/bin/bash •() { IFS=";" printf "\e[%sm" "${@:-0}"; }; [[ -t 1 ]] \|\| •() { :; } [[ "$(pwd)" == "/" ]] && root="/" \|\| root="" none=$(•) IFS=$'\n'; for line in $(grit.rb); do stat=${line:0:2} rest=${line:3} case $stat in # NOTE: codes should be TWO digits so columns line up! "A ") tint=$(• 32) ;; # added is green " D") tint=$(• 31) ;; # deleted is red " M") tint=$(• 33) ;; # modified is yellow " ") tint=$(• 37) ;; # unchanged is white ) tint=$(• 36) # others are cyan esac echo "${tint}${stat}•${root}${rest}${none}" done \| column -t -s • Last edit by shreeve* at 15 Jan 2023, 10:52 AM -------------
	[top]

shreeve	Posted: 15 Jan 2023, 10:54 AM
Member Posts: 4 Joined: 10-January 23	And here's the new `grit.rb` code: #!/usr/bin/env ruby list = {} note = {} last = {} hash = nil char = "•" args = "--oneline --name-status --date=format:'%Y-%m-%d %H:%M:%S' --format='%h %cd#{char}%s' . \| grep ." Dir.chdir `git rev-parse --show-toplevel`.chomp `git ls-files` .split("\n").each {\|e\| list[e ] = ' ' } `git status -suno .`.split("\n").each {\|e\| list[e[3..]] = e[0, 2] } `git log #{args}` .split("\n").each {\|e\| e.split(/\s+/,2); $& == " " ? note[hash = $`] = $' : last[$'.split("\t",2).last] \|\|= hash } list.each {\|path, stat\| puts [stat, path, hash=last[path], note[hash]] * char } I just wanted to follow up and post it here so that it's somewhere out there. I also haven't been able to find anything on Google or elsewhere describing how github or gitlab do this commit history listing. There are actually some pretty tricky concepts in there, but I think this is a great step forward. Thanks again for your initial work and ideas. Cheers, Steve -------------
	[top]

mit	Posted: 18 Jan 2023, 04:59 PM
yeah whatever Admin Posts: 566 Joined: 4-May 16	Not a bad effort! If you really wanted to speed it up, and I suspect this is how github and gitlab do it, you should forget the git binary entirely and just walk the history yourself. It may even be possible to fork the git source code and add your grok function. Doing it in C should make it at least as fast as a single git log call. -------------
	[top]

meck	Posted: 11 Mar 2023, 11:40 AM
Member Posts: 1 Joined: 11-March 23	@mit Came here as I found your blog entry on dotfiles very nice and consider using it myself. I see you also keep files outside your home directory in your repo, e.g. from the /etc dir. But many files outside your home have a different owner and some are only readable by root. Maybe I've missed it but how do you deal with that? Do you use sudo to add those files? If so I'd be worried that I get pretty messy permissions in the .git directory and that this would throw errors whenever I do things like checking out a previous version which can only be solved by sudo again. -------------
	[top]

mit	Posted: 11 Mar 2023, 01:22 PM
yeah whatever Admin Posts: 566 Joined: 4-May 16	That's a good question! In practice it hasn't been an issue. The vast majority of files are world-readable so they can be tracked without a problem. Yes, it would throw a permission error if I tried to checkout a different version of it, but that doesn't come up very often. In real usage, seeing the diff is enough for me to make my mind up about what to do, and then I can just sudo -E vim to resolve it. If I find that doing a system upgrade has changed one of my tracked files, sometimes I restore it, but often it's a sign that I'm configuring it in the wrong place. In most cases there's more than one way to configure it, e.g. a lot of things have a conf.d where you can chuck stuff that won't get overwritten. The only files that can't be tracked without sudo can be listed like this: sudo find /etc ! -perm -o=r It's pretty much just /etc/shadow, GPG keys, NetworkManager WiFi passwords, bunch of stuff I have no interest in tracking. There's CUPS config too, which I'm not tracking. But it is group readable, so if I wanted to track it I would just add myself to the group for it. In the most extreme case where I couldn't track it all, the EFI boot config, I just made a text file (/boot/efi.txt) and wrote some notes on how I configured it. I'm quite pleased you can boot arch using EFI stub, no need for grub. It's the kind of thing I only ever look at every few years and always forget the details, so I'll be glad to have those notes if I need to fiddle with it in future. -------------
	[top]