Esteve Personal Blog

Our freedom in the desktop/notebook world has always been threatened, but we have free operating systems that can compete with any closed solution, however when it comes to the phone industry we are mostly jailed to closed systems, specifications are rarely published and there does not seems to be any hope that this will change.

After seeing Pof’s smartphone ( Blueangel ) running Linux I was so amazed that decided to start looking for a nice one capable of running a decent Linux. What follows is a list of the smartphones that have a GSM phone, and for which exists some kind of free operating system port, the list is not complete and is based on Linux kernel phones, please leave comments on phones not included here I hope there is a phone which I don’t know that can run a free OS perfectly.

(more…)

This weekend I’ve been working again on the iPod Shuffle randomness… I’ve been quite busy lately, that is why I haven’t been posting.

In this second post I will be finally using an iPod Shuffle and will show the first results on it’s degree of randomness, this results are probably
not statistically correct, as I don’t have enough samples yet…

The Analyzer /dev/dsp

As you might remember in last iPod post I gave a program that converted some WAV files to a number by analyzing it’s content, now I modified this program
by taking the data from /dev/dsp insted that from a WAV file.

Data is taken in small chunks, the FFT is done on this chunk, when the FFT is has been finished I search for the frequency peak of the sample. This peak
is used to determine which tone was in the original file.

The test set

I’ve filled the iPod with exactly 256 audio files, this number of files is exactly 1 byte, we will see why I’m using this number of files later. Each of
this files has an Id number which is a prime number so the id of the file ranges from 2 to 1619 ( the first 256 prime numbers ), this numeration is used as error
detection. Here is a sample of the number 1531 and 1123, if anyone needs the full
set just post a comment and I’ll be glad to send it.

The first results

After uploading the files to the iPod the test can begin, the output of the Shuffle is plugged into the Mic input of the pc. The iPod is put in shuffle position,
the analyzer program is started and then the play button on the iPod is pressed ( iPod volume has to be put to the maximum ). Some hours later, when the Shuffle’s
battery has been emptied ( 8 hours or so ), we can have a look at the output file. The file contains 9322 numbers ranging from 2 to 1619. This numbers have
not been checked against errors.

Text output to stream

It’s time to check the previous output for errors and to convert it to a stream for further processing, this binary stream will be passed to a program
that analyzes it’s randomness. The range 2-1919 is converted to a range of 0×00-0xFF ( 1byte ), we are using 1 byte values as the randomness analyzer
takes 1 byte values. The range conversion is done by assigning a number to each prime, for example 2 is 1, 3 is 2, 5 is 3, … , 71 is 20, … , 809 is 140, … and
1619 is 255. If any of the values of the input file can not be converted to the range is discarded and is considered as a detection error, I’ve not
implemented error correction, but it will be done later.

Randomness

At this point we have a binary file with the random data obtained from the iPod, we pass this output to ENT, this
program applies a series of tests to a stream of bytes, this tests help determine the randomness of this sequence.

A full example

Now I’ll show the process and results obtained from this first test. At the end of this post all source code for the programs I’ve used can be obtained, of
course this sources are GNU/GPL.

First of all we generate the files that will be uploaded to the iPod Shuffle, this script uses the min_gen_number.sh script to generate wav files,
and then convert them to MP3 files:

# ./whole_set.sh

Those 256 files are uploaded to the Shuffle using gtkpod or anything else… Now we need mini-jack to mini-jack cable to connect the iPod’s
headphone output to the Mic input of the computer. iPod’s volume has to be high when playing back files, also remember to put the iPod in Shuffle position.

# ./analyze_dsp "out$(date).txt"
( 8 hours later )
# cat "outSun Sep  4 21:39:14 CEST 2005.txt" | wc -l
9322

Now we have a file called outSun Sep 4 21:39:14 CEST 2005.txt, this file contains 9322 numbers
which can contain errors. Now we have to transform this output to binary, and with the range of 0-255.

# ./streamize "outSun Sep  4 21:39:14 CEST 2005.txt" Sunday.bin
ERRROR 255
ERRROR 525
ERRROR 489
ERRROR 485
ERRROR 235
ERRROR 803
# ls -l Sunday.bin
-rw-r--r--  1 esteve users 9316 Sep  5 12:21 Sunday.bin

A binary file Sunday.bin has been created, as we can see it has detected some errors, this numbers have been discarded.
The size of the file is 9322 - 6, where 6 are the number of errors. Now this binary file is used as input to ENT:

# ./ent Sunday.bin
 Entropy = 7.999828 bits per byte.

 Optimum compression would reduce the size
 of this 9316 byte file by 0 percent.

 Chi square distribution for 9316 samples is 2.22, and randomly
 would exceed this value 99.99 percent of the times.

 Arithmetic mean value of data bytes is 127.4317 (127.5 = random).
 Monte Carlo value for Pi is 3.110824742 (error 0.98 percent).
 Serial correlation coefficient is 0.018126 (totally uncorrelated = 0.0).

We can compare this result with the result obtained by suppling ENT with a true random file obtained from timing radioactive decay events at
Hotbits:

# ./ent Hotbits
 Entropy = 7.979351 bits per byte.

 Optimum compression would reduce the size
 of this 8192 byte file by 0 percent.

 Chi square distribution for 8192 samples is 232.50, and randomly
 would exceed this value 75.00 percent of the times.

 Arithmetic mean value of data bytes is 126.8322 (127.5 = random).
 Monte Carlo value for Pi is 3.129670330 (error 0.38 percent).
 Serial correlation coefficient is -0.016542 (totally uncorrelated = 0.0).

Sources

• whole_set.sh
• min_gen_number.sh
• analize_dsp.c, requires fftw3 and compile options : -lfftw3 -lm
• streamize.c

Update: By the way this Blog was one year old on the 2nd of september!! I forgot to post it.

First of all I must apologize for not having written anything on the blog for so long, I haven’t been much creative lately, and I have had little time to spend on writing, I hope this summer will be more productive.

iPod shuffle has been one of the latest revolutionary products from Apple, it aims to listen to your music collection randomly. But how random is it ? My objective will be to evaluate how random songs are played, thus knowing a little more about the random number generator inside the shuffle.

First Implementation (No iPod used, yet):

The idea is to fill the iPod with a collection of mp3 files, connect it’s header output to the computer’s microphone line, let the iPod play the files and recognize each one, this will give a list of the shuffled files. Once this list is big enough it’s order can be studied to recognize playing patterns and evaluate how random it is.

Encoded MP3 files

The iPod will contain a set of mp3 files that will be easy to recognize when played back to the computer. An identification number will be given to each file, this number will be encoded as audio and then compressed to mp3.

To encode the number we will be using frequency modulation. For this first implementation I will be using a not very efficient but easy to recover modulation. The identification number will be converted to binary and then the audio file will be generated as follows:

• Insert a tone at FREQ_NEW hz.
• For each bit of binary id. number starting at MSB do:
• • Insert a tone at FREQ_SEPARATION hz
  • If the bit is 0, Insert a tone at FREQ_ZERO hz
  • If the bit is 1, Insert a tone at FREQ_ONE hz

To generate each tone I will use a sine at the specified frequency, to generate them I found a tool called sgen from siggen set of tools. This tool lets generate through command line raw audio files containing a single tone at a specified frequency. To generate each file from a number I used a shell script that receives as parameter the Id number and uses sgen to create an audio file.

As an example we will create the id audio file 91. The 91 to binary is 1 0 1 1 0 1 1, so the end file will contain the following tones:

• NEW
• SEPARATION
• ONE
• ZERO
• ONE
• ONE
• ZERO
• ONE
• ONE

As it can be seen this is not very space effective but is really easy to implement and to recover.

The script can be found here, it only requires siggen and lame.

Recognize each file :

Previous to do any attempt with a real iPod I implemented a program that recognizes each file. The input to this program is a WAV file with a number encoded inside as described above, the program will analyze the input and then output the number. This same technique will be used to recognize each file the shuffle is playing.

The objective is to be able to recognize each tone, this will be accomplished doing a Fourier transform of a sample of the file and try to detect a predominating frequency, once the frequency of each tone is recognized the conversion to a number is direct ( see encoding above ).

To perform the FFT I have used the fftw libraries. I go through the input file sampling chunks of it, and then detecting the main frequency from each.

I found this interesting example on how to use the fftw libraries to plot a spectrogram from a sample.

A first version of the program can be found here , it does well the recognition of files, I have tested adding noise to each of the files and seems to perform well, I will show the tests later on this post. The program needs the fftw to compile and expects an encoded wav file as input.

The File Set:

I have generated a set of about 500 files that will be uploaded to the iPod, these files have been generated with the gen_number.sh script and they have the particularity that each ID number is a prime number, this way when recognizing back each file, if the recognition returns a non prime number there is an error and the number should be discarded.

To test that the decoder program works well each of this file has been decoded back to its id number, none of the 500 files has failed the test.

From Euro Telcoblog I’ve read Mapping VoIP, another perspective a comment on a voipuser forum entry that gives usage numbers from outbound voip calls.

But what I found to be most interesting is the follow-up comment of Dean:

I checked this against our website logfiles and the demographic of our members and users. Unsurprisingly, they’re an almost perfect match [ NOTE from Esteve: This refers to the first entry of the forum, where a list of conuntries and outbound minutes has been given ] .

What does this mean? Well, it means that our members (some 8,000 of you in all) don’t make many international calls. Most of you are making phone calls within your own country.

What does that say about VoIP? Not much beyond what we’ve already been talking about anyway - VoIP is not a cheap bucket-call-shop phone replacement service (the traditional cut price telecoms companies made their money on international routing). You’re using it to test out your asterisk boxes, re-routing facilities, voicemail to email etc.

I’ve always felt that VoIP is about adding value to telephony, not cutting prices. These facts and figures certainly don’t seem to me to contradict that.

Dean

It’s easy to use a Linux as an iTunes server, I’ve just tried so and works perfect. First I installed rendezvous . After that I emerged daapd. Then it is necessary to edit /etc/daapd.conf so the Root points to your music directory ( ie: /home/music ). After that as root we can launch the service ( being 192.168.1.121 your iTunes daapd server ):