danieljon.es

normal solarized dark no css
index blog opinions interesting cgit
likes dislikes

Blog

Rambles about programming, technology and life.

prev 1 2 3 4 5 6 7 8 9 next
direct link

Bit manipulation

3/11/2018

Bit manipulation is something I find interesting, to strengthen my understanding I thought it'd be best to put some of the basics in text.

First, an understanding of binary is required:

128
64 32 16 8 4 2 1
0 0 1 0 1 1 0 0
= 44

Got it? I knew you would. If not see my post on binary.

Next you need an understanding of bitwise operations:
AND (&)
copies bit if it is set in both operands
11 AND 5 (1011 AND 0101) = 1
1011
0101
----
0001
OR (|)
inclusive OR, copies bit if it is set in either operand
13 OR 7 (1101 OR 0111) = 15
1101
0111
----
1111
XOR (^)
exclusive OR, copies bit if it is set in either operand but not if it is set in both
13 XOR 7 (1101 XOR 0111) = 10
1101
0111
----
1010
NOT (~)
flips each bit obtaining the numbers one's complement
NOT 5 (101) = 2
101
---
010
Left Shift (<<)
Shifts the bits of the left operand by the number in the right operand to the left
5 << 2 (101 << 010) = 20
000101
 <<<	(the 101 is moved to the left by 2 bits)
------
010100
Right Shift (>>)
Shifts the bits of the left operand by the number in the right operand to the right, underflow is discarded
22 >> 3 (10110 >> 00011) = 2
10110
   >>>>>(the 10110 is move to the right by 3 bits)
-----
00010
Now that those are understood, let's take a look at some bit manipulation, I'm going to use C bitwise operators from now on.

First note, counting bits starts from the least significant bit (right-most), which is the 0th bit.
Setting a bit
To set a bit to 1 OR the number with 1 left shifted with the desired bit to set
number |= (1 << bit)
number = 5, bit = 3
5 | (1 << 3) = 13 

1 << 3 = 8
 0001
 <<<
-----
 1000

5 | 8 = 13
  0101
| 1000
------
  1101
  ^
  our 3rd bit (0,1,2,3) is now set
Clearing a bit
To clear a bit AND the number with the one's complement of 1 left shifted with the desired bit to clear
number &= ~(1 << bit)
number = 9, bit = 3
9 & ~(1 << 3) = 1

1 << 3 = 8
0001
<<<
----
1000

~8 = 7
  1000
~
------
  0111

9 & 7 = 1
  1001
& 0111
------
  0001
  ^
  our 3rd bit (0,1,2,3) is now 0
Getting a bit
To get a bit right shift the number by the desired bit to retrieve, then AND it by 1
result = (number >> bit) & 1
number = 14, bit = 3
result = (14 >> 3) & 1
result = 1

14 >> 3 = 1
1110
>>>
-----
0001

1 & 1 = 1
01
01
--
01
Toggling a bit
To toggle a bit XOR the number by 1 left shifted by the desired bit
number ^= 1 << bit
number = 12, bit = 4
12 ^= 1 << 4 = 28

1 << 4 = 16
00001
<<<<
-----
10000

12 ^ 16 = 28
  01100
^ 10000
-------
  11100
  ^
  our 4th bit (0,1,2,3,4) is now set
Let's do something with this knowledge and use the bits of a char as individual flags that we can use. Using an enum is a much better way to do this.
#include <stdio.h>

int
dectobin(int dec)
{
	/* return the binary representation of dec */
	if (dec == 0) return 0;
	if (dec == 1) return 1;
	return (dec % 2) + 10 * dectobin(dec / 2);
}

int
getbit(int num, int bit)
{
	return (num >> bit) & 1;
}

int
togglebit(int number, int bit)
{
	return number ^= 1 << bit;
}

int
setbit(int number, int bit)
{
	return number |= 1 << bit;
}

int
clearbit(int number, int bit)
{
	return number &= ~(1 << bit);
}

int
main(void)
{
	unsigned char flags = 0; /* char because it is 1 byte, unsigned because we don't need negatives */
	printf("flag has value %d, in binary: %08d\n", flags, dectobin(flags));
	puts("let's make the number 145");
	puts("we need to toggle bits: 0, 4, 7");
	flags = setbit(flags, 0);
	puts("set bit 0");
	printf("flag has value %d, in binary: %08d\n", flags, dectobin(flags));
	flags = setbit(flags, 4);
	puts("set bit 4");
	printf("flag has value %d, in binary: %08d\n", flags, dectobin(flags));
	flags = setbit(flags, 7);
	puts("set bit 7");
	printf("flag has value %d, in binary: %08d\n", flags, dectobin(flags));

	puts("let's check the value of bit 7");
	printf("bit 7 is %d\n", getbit(flags, 7));
	puts("we don't want bit 7 set anymore");
	flags = clearbit(flags, 7);
	puts("cleared bit 7");
	printf("flag has value %d, in binary: %08d\n", flags, dectobin(flags));
	return 0;
}

output:
flag has value 0, in binary: 00000000
let's make the number 145
we need to toggle bits: 0, 4, 7
set bit 0
flag has value 1, in binary: 00000001
set bit 4
flag has value 17, in binary: 00010001
set bit 7
flag has value 145, in binary: 10010001
let's check the value of bit 7
bit 7 is 1
we don't want bit 7 set anymore
cleared bit 7
flag has value 17, in binary: 00010001

direct link

string replacement in files

14/10/2018

I'm slowly rewriting my website generator in C and for it I required an easy way to write an input file to an output file replacing every instance of something with another string. So I made this convenience function that might be useful elsewhere one day.

int replaceinfile(char *originalfile, char *destinationfile, char *placeholder,
								char *replacement);
// 0 = failure, 1 = success

replaceinfile("input.txt", "output.txt", "%PLACEHOLDER%", "replacement string");
input.txt
Normal text file

%PLACEHOLDER%
%PLACEHOLDER%
%PLACEHOLDER%

With lines and whatever, %PLACEHOLDER%

123%PLACEHOLDER%123	
ok %PLACEHOLDER% ok
output.txt
Normal text file

replacement string
replacement string
replacement string

With lines and whatever, replacement string

123replacement string123	
ok replacement string ok
The source is available in my git repository.

direct link

startup scripts

6/10/2018

I setup and open a few things when I log into my computer through .bash_profile, these include two instances of irssi, mutt, a slack-to-irc gateway and cmus.

This is achieved through three shell scripts:

main.sh - this script launches my other two scripts. It checks for the file /tmp/started, if it exists nothing is done and the script exits. If the file does exist the file /tmp/started is created (empty) and my two other scripts start. /tmp is wiped on boot.

#!/bin/bash

cd "$HOME/programming/bash/autostart"

if [ ! -f /tmp/started ]; then
	TMPFILE="/tmp/started"
	touch $TMPFILE
	./startup.sh
	./slack.sh
fi
cd $HOME
startup.sh - This script is responsible for launching irssi, cmus and mutt. I open these in a tmux session named main. It has 3 windows named irssi. cmus and mutt respectively.
#!/bin/bash

tmux new-session -d -s main -n irssi
tmux new-window -t main:1 -n cmus
tmux new-window -t main:2 -n mutt

tmux send-keys -t main:0 "firejail irssi" C-m
tmux send-keys -t main:1 "cd $HOME/music/japanese;cmus" C-m
tmux send-keys -t main:2 "firejail mutt" C-m
cd $HOME
slack.sh - This script is responsible for launching my slack-to-irc gateway and an irssi instance that automatically connects to it. Once again a tmux session is created with two windows named lsirc and irssi.
#!/bin/bash
BASE="$HOME/programming/python/localslackirc"
cd $BASE
tmux new-session -d -s slack -n lsirc
tmux new-window -t slack:1 -n irssi

tmux send-keys -t slack:0 "cd $BASE;python3 ./irc.py -j" C-m
sleep 1
tmux send-keys -t slack:1 "firejail irssi --home $HOME/.slackirssi -c 127.0.0.1 -p 9007" C-m
While simple, its really effective and saves a lot of effort every boot. Is there a better way to run a script once when you initially log in? Probably but the effort to search was more than it was worth.

direct link

cfg parser

5/10/2018

I wanted to roll my own cfg/ini-like parser in C, it started off somewhat elegant to my low standards, however it quickly became a mess. It isn't great, but it "works".

It handles simple cfg/ini like files with 'key=value' pairs.

It's really hacky, I wouldn't really trust it for anything.

Comments are ignored, they're denoted by the '#' character.

Spaces are currently treated literally. test=123 and test = 123 are DIFFERENT, the key has a space appended and the value a space prepended in the second example.

Duplicate keys are not handled currently, the cfggetvalue function will only give you the first value.

See settings.cfg for an example cfg file.

To compile type make. Run the example program by issuing ./example.

All functions return 0 on failure, this goes for getting a value, always check 0 isn't returned. If it is, that means the key was not found.

Writing back to the cfg file is NOT implemented.

main.c has a usage example, however the usage structure is like this:

// allocate memory for the cfg structure
struct cfgfile *cfg = malloc(sizeof(struct cfgfile));
// setup cfg structure passing it the cfg structure we just made and a file name
cfgsetup(cfg, "filename.cfg");

// create a buffer for our value
char buffer[256];
// collect our value, the function returns 0 on a failure (when it cannot find the key)
int val;
val = cfggetvalue(cfg, "key", buffer, sizeof(buffer));
if (val != 0)
	puts(buffer); // print our value
else
	puts("cannot find the key!");

// free all of our memory (including cfg itself)
cfgfree(cfg);
The source is available in my git repository.

direct link

Binary clock

19/9/2018

To force me to learn recognising binary numbers easier I've created a small program that takes the current time and converts it to binary. I apply this binary string to the X displays root window title, dwm takes this string and puts it into the status bar, it replaces the decimal clock that was there before. Now if I want to know the time I have to convert it into decimal or spend the effort typing 'date' into a terminal.

The source is available in my git repository.


direct link

What am I doing?

18/8/2018

I don't know what I'm doing with my life. It's going no where and I'm up against a brick wall I just can't seem to get myself over, no matter how much I try.

At what point is it fair to give up?


direct link

x86 Assembly

16/7/2018

A little while ago I temporarily became interested in 8085 assembly and wrote a few little exercises and quickly lost interest. However, the basics were retained and I was highly interested in some day coming back to it.

Over the last few days I have been looking at x86 assembly (Intel syntax, NASM assembler) and have followed various tutorials and videos furthering my existing, yet basic knowledge. At this point I felt I was comfortable enough to attempt a small and simple exercise, I chose to generate Fibonacci numbers.

First, I wrote the program in C to understand the required logic flow.

#include <stdio.h>

int main(void)
{
	int i, tmp;
	int first = 0;
	int second = 1;
	printf("%d\n%d\n", first, second);
	for (i = 0; i < 10; i++)
	{	
		tmp = second;
		second += first;
		first = tmp;
		printf("%d\n", second);
	}
	return 0;
}
Using the C version, I translated the program into x86 assembly:

fibonacci.asm
%include 'functions.asm'

SECTION .data
MAX db 8	; 10 - 2 (we print those outside the loop) 

SECTION .text
global _start

_start:
	; eax -> first
	; ebx -> second
	; ecx -> counter
	; edx -> tmp
	; MAX -> maximum iterations
	mov eax, 0
	mov ebx, 1
	mov ecx, 0
	mov edx, 0

	; print the first iteration of values (0, 1)
	call printintlf
	push eax	; store eax on the stack
	mov eax, ebx 	; printintlf requires the int in eax
	call printintlf
	pop eax		; restore eax
loop:
	inc ecx	
	mov edx, ebx 	; store second as tmp data	
	add ebx, eax 	; add first to second
	mov eax, edx	; set first as what second was (tmp variable)
	push eax	; store eax on the stack
	mov eax, ebx	; printintlf requires the int in eax
	call printintlf
	pop eax		; restore eax
	cmp ecx, [MAX] 	; check if our counter has reached MAX, if it hasn't loop
	jne loop

	call exit
'functions.asm' is a file containing some standard functions for string/int printing, exiting etc. I wrote the functions following the tutorials at asmtutor.com. You can find the file here if you care to see it.

I hope to spend considerably longer learning and writing programs this time.

direct link

Backup procedure

14/6/2018

A year or two ago I my bulk data storage drive died - it was 2TB and held information spanning back to around 2013 when I first got this computer. Of course, none of the data was backed up. I thought this would hurt, but it didn't. I've not thought twice about the data I had lost. I guess none of it was important.

Today, things are a little different. I keep semi up-to-date copies of things that are important to me (namely my anime collection and /home) on an external 1TB drive that I always have on my body. My thought process here is if both the drive I carry and my computer are destroyed at the same time, I'd probably be dead too.

I also keep a similarly semi up-to-date backup of my server on the same drive. This backup is never as up-to-date as I'd like.

I use cronjobs and bash scripts to perform daily backups of my personal computer and server. These are stored on a drive dedicated to backups always mounted on my computer, in the case of my server - a mere directory. Every now and then I copy these backups to the drive I carry everywhere. These are single tar files gzip compressed. I completely understand having the backups mounted and accessible is not the right way to do things.

I keep these backups for a short period after which they are deleted and replaced with newer ones.

Local personal computer backup cronjob and script:

# cronjob
# runs everyday at 1pm
0 13 * * * /home/daniel_j/programming/bash/backup/backup.sh
# delete backups older than 5 days
# runs every day at 3pm
0 15 * * * find /mnt/backups/tar_backups/old_backups/ -type f -mtime +5 -delete

# backup script
#!/bin/bash

# move the last backup performed into the old backups directory
mv /mnt/backups/tar_backups/*.tar.gz /mnt/backups/tar_backups/old_backups/
#backups
tar -cvpzf /mnt/backups/tar_backups/home-backup-$( date '+%Y-%m-%d_%H-%M-%S' ).tar.gz /home  > /dev/null
echo "buzz=500" >/dev/tcp/localhost/3001
(As a side note, the "echo "buzz" > /dev/tcp/localhost/3001" sends a command to an arduino that sounds a buzzer. I use it as an alert. I'll write more about this in another post.)

Server:
# cronjob
# backup
@daily /home/username/scripts/backup.sh
# delete backups older than 2 days
@daily find /home/username/backups/old_backups/ -type f -mtime +2 -delete

# backup script
#!/bin/bash

# move the last backups performed into the old backups directory
# (I backup both directories and an sql dump)
mv /home/username/backups/*.tar.gz /home/username/backups/old_backups/
mv /home/username/backups/*.sql /home/username/backups/old_backups/

# backups

# dump sql databases
/usr/bin/mysqldump --all-databases > /home/username/backups/dump-$( date '+%Y-%m-%d_%H-%M-%S' ).sql -u root -pr00tpassw0rd

# I backup everything valuable on /
tar -cvpzf /home/username/backups/backup-$( date '+%Y-%m-%d_%H-%M-%S' ).tar.gz --exclude=/home/username/backups --exclude=/proc --exclude=sys --exclude=/mnt --exclude=/media --exclude=/run --exclude=/dev --exclude=/var/www/desu/f --exclude=/home/username/old_server --one-file-system /  > /dev/null
As for my anime collection, that is simply an rsync command.

direct link

Anime tracker

8/6/2018

MAL (MyAnimeList) recently forced every user with an account to reset their password, they say this is out of caution regarding an exploit they found within their API. A few days later the entire website disappears for days, completely inaccessible. They gave no explanation for this and weeks later many of their services are still unavailable.

Clearly, this is beyond acceptable. Through this disaster however came a new project idea.

I've began work on creating my own anime static anime tracker. You can find the goals of the project here.

There is a live version of the project here.
The source code can be found here, licensed under GPL2.


direct link

Miscellaneous RGB controller additions

13/4/2018

A month or two back I obtained a monitor arm in order to further reduce the clutter on my desk. Once setup I decided to get my RGB controller project back in a working order.

I ended up putting the strip to the underside of my monitor as well as mounting the arduino and accompanying breadboards on the back.

I added an oled display on the bottom of my monitor which currently displays the time and current song time/duration. This is achieved with a simple bash script. The arduino was programmed accordingly to receive and display the data. I also added a switch that controls the power going to the LED's. There is a slight leak of power when the RGB channels are set to 0 (meaning off), this switch allows me to kill the power supply entirely. Finally, I added a female power jack to make supplying the required 12v easier.

The code as always is available on git, now licensed under GPL 3.

The wiring is an absolute hackjob. I don't particularly enjoy wiring or working with components, let alone a soldering iron.


prev 1 2 3 4 5 6 7 8 9 next



source

page generated 2018-11-05 11:52:17 using sitegenerator