Making the RubyGem work (bundle & rake install) with Windows - ruby for different platforms!

Ruby file .rb:
Detecting operating systems on Ruby, the backslashes are for regex:
    Linux: /linux/
    Windows: /win32/
    Mac OS: /darwin/
So a ruby file would have these things to distinguish different code for different platforms:

case RUBY_PLATFORM
  when /i386-mingw32/ # OR /win32/ matching Gemfile.lock under PLATFORMS
    require_files :win32
  when /linux/
    require_files :linux

else
    raise "Unsupported platform #{RUBY_PLATFORM}"
end

Gemfile:
Gem Bundler's Gemfile has special syntax (documentation, and examples).

Here is an example to write Windows-only gem:

### windows specific gems ###
 gem 'win32-process', :platforms => [:mswin, :mingw]
 gem 'win32console', :platforms => [:mswin, :mingw]

or 

### windows specific gems ###
platforms :mswin, :mingw do
    gem  'win32console', '1.3.0'
    gem  'wind32-process'
end

The symbols ":mswin" and ":mingw" are special platforms groups.

gem "rspec", :group => :test
gem "wirble", :groups => [:development, :test]

Unfortunately, there is no ":linux", so it'll be a non-platform group, which means the command should be "bundle install --without linux" when running on Windows.

Yay! Can:

bundle install --without linux
rake install

commit 8b8573dcc87e673cca042e2d68569fffd586e872
try if Windows rake will work after making system/getifaddrs only for linux


====
Next step:
- downgrade to 1.8.7 ruby
- make sockets work  (google, more wireshark digging)
- How to list devices, look at: fad-win32.c.
(change :linux to :ruby to avoid "--without linux" on Windows, i.e. does :ruby work as platform thingy not just a group?)
===
IRC: freenode ruby chat
#RubyOnRails
#bundler
#ruby-lang

==
This pipe server Windomit ws gem be helpful?

todo

0. Can I incorporate system-getifaddr?

• Yes, and add "system/getifaddrs" to the list of required gems. No need for requiring rubygems

1. What's wrong with ethernet_ping? Solved
2. Why can't I rspec? Fixed
3. list devices (all os)
- add in the mac address listing thing
- fix the require system/getifaddrs
- make it possible to rake install on windows!! (Gemfile has special syntax, rest of "system-ifaddrs" just regex).
- downgrade windows ruby from 1.9.2 to 1.8.7


- add enable_pcap, ether_ping_server

Listing all network interfaces/devices and their MAC Addresses (for Linux with Ruby)

I used this gem to get System.get_ifaddrs() working on Ruby.

(Testing in ether_shell because my ethernet still can't pass rspec.)

For the 'system/getifaddrs' require, have to let user install it before they can use it, so do:
1. Add into the Gemfile, like

  2 # Add dependencies required to use your gem here.
  3 # Example:
  4 #   gem "activesupport", ">= 2.3.5"
  5   gem "system-getifaddrs", "~> 0.1.1"

2. Do NOT add gem.add_runtime_dependency 'rcov', '>= 0.1.1' to Rakefile because it's automatic now
3. bundle
4. sudo rake install

Add in lib/ethernet/raw_socket.rb:

require 'system/getifaddrs' # for listing

...

  # returns a symbols array of all network devices
  def self.list_all_devices()
    case RUBY_PLATFORM
    when /linux/ #TODO: test to see if works on MacOS as well 
      devicehash = System.get_ifaddrs # {:eth0=>{:inet_addr=>"18.248.7.89", :netmask=>"18.248.7.89"}, :vmnet1=>{:inet_addr=>"172.16.76.1", :netmask=>"172.16.76.1"}}
      devicehash.keys
    end 
  end 

  # returns a hash of all network devices => MAC Address
  def self.list_all_devices_macaddr()
    case RUBY_PLATFORM
    when /linux/ 
      devices = self.list_all_devices
      device_macaddrs = {}
   
      devices.each do |eth_device|
        device_macaddrs[eth_device] = self.mac(eth_device.to_s).unpack('H*')[0]
      end 
      device_macaddrs
    end 
  end 

Add in spec/ethernet/raw_socket_spec.rb

describe 'testing listing all network devices' do
     it 'should list the devices' do
       device_macaddrs = Ethernet::RawSocket.list_all_devices_macaddr()
       device_macaddrs.each do |key, value|
         puts "\nDevice is: "
         puts key.to_s
         puts "MAC Addrs: "
         puts value
         puts "\n"
       end 
     end 
  end 

Results:

$ sudo rspec raw_socket_spec.rb 
.....
Device is: 
vmnet1
MAC Addrs: 
005056c00001


Device is: 
vmnet8
MAC Addrs: 
005056c00008


Device is: 
lo
MAC Addrs: 
000000000000


Device is: 
eth0
MAC Addrs: 
002170925bc5

.

Finished in 0.11353 seconds
6 examples, 0 failures

Details of how I found this solution, very boring

I could run ether_shell's specs. Specifically, sudo rspec raw_socket_spec.rb, which currently has eth0 as the device

3 describe 'RawSocket' do
  4   let(:eth_device) { 'eth0' } 

Now I want to see if the outputting mac address function works.

In raw_socket.rb, .mac() is the function that gets the mac address of a device:

28   # The MAC address for an Ethernet card.
 29   #
 30   # Args:
 31   # eth_device:: device name for the Ethernet card, e.g. 'eth0'
 32   def self.mac(eth_device)
 33     case RUBY_PLATFORM 
 34     when /linux/
 35       # /usr/include/net/if.h, structure ifreq
 36       ifreq = [eth_device].pack 'a32'
 37       # 0x8927 is SIOCGIFHWADDR in /usr/include/bits/ioctls.h
 38       socket.ioctl 0x8927, ifreq
 39       ifreq[18, 6] #in scratchpad's raw_ethernet, then says .unpack('H*').first
 40     else 
 41       raise "Unsupported platform #{RUBY_PLATFORM}"
 42     end
 43   end

I changed it so that it prints out the mac: (line numbers don't match)

28       # /usr/include/net/if.h, structure ifreq
 29       ifreq = [eth_device].pack 'a32'
 30       puts "\ndevice name: "
 31       puts ifreq
 32       # 0x8927 is SIOCGIFHWADDR in /usr/include/bits/ioctls.h
 33       socket.ioctl 0x8927, ifreq
 34       puts "mac address output: "
 35       puts ifreq[18, 6].unpack('H*')[0]
 36       puts "-----------"
 37       ifreq[18, 6]

$ sudo rspec raw_socket_spec.rb

device name:
eth0
mac address output:
002170925bc5
-----------
.
device name:
eth0
mac address output:
002170925bc5
-----------
..
device name:
eth0
mac address output:
002170925bc5
-----------
..

Finished in 0.30495 seconds
5 examples, 0 failures

yup, the mac address of my eth0 is indeed that. (matched HWaddr in ifconfig -a of eth0)

the ".unpack('H*')[0]" part changes the mac address into hex, without it, it would look like:
!p�[


hmmm.... so let's see if it gets the right mac address if I change the device to vmnet1
so, raw_socket_spec.rb:

3 describe 'RawSocket' do
  4   let(:eth_device) { 'vmnet1' }

$ sudo rspec raw_socket_spec.rb

device name:
vmnet1
mac address output:
005056c00001
-----------
.
device name:
vmnet1
mac address output:
005056c00001
-----------
..
device name:
vmnet1
mac address output:
005056c00001
-----------

a match! yipee!!! (the rspec hung, but that we don't care)

Also worked for wlan0, even though I'm not plugged in to it

device name:
wlan0
mac address output:
001c26640fac
-----------


============================================

So now, I need just get the list of names and then run .mac() for each of them.
I need to look at the code for systemgetifaddr

man 3 getifaddrs
http://comments.gmane.org/gmane.os.cygwin/123705 ioctl vs getifaddrs

The rspec of https://github.com/bbcoimbra/system-getifaddrs/blob/master/spec/system-ifaddrs_spec.rb compares it to ifconfig's result

added to raw_socket.rb:

2 require 'rubygems' # for listing
  3 require 'system/getifaddrs' # for listing
...
 45   # listing all network devices's MAC addresses
 46   def self.list_all_devices()
 47     case RUBY_PLATFORM
 48     when /linux/ #test to see if works on MacOS as well 
 49       devicehash = System.get_ifaddrs # {:eth0=>{:inet_addr=>"18.248.7.89", :netmask=>"18.    248.7.89"}, :vmnet1=>{:inet_addr=>"172.16.76.1", :netmask=>"172.16.76.1"}}
 50       devicehash.each_key do |key|
 51         puts self.mac(key.to_s).unpack('H*')[0]
 52         puts " - - - - -"
 53       end
 54     end
 55   end

added to raw_socket_spec.rb:

42   describe 'testing listing all network devices' do
 43      it 'should list the devices' do
 44          EtherShell::RawSocket.list_all_devices().should_not be_empty
 45      end
 46   end

good results:

$ sudo rspec  raw_socket_spec.rb 

device name: 
eth0
mac address output: 
002170925bc5
-----------
.
device name: 
eth0
mac address output: 
002170925bc5
-----------
..
device name: 
eth0
mac address output: 
002170925bc5
-----------
..
device name: 
vmnet1
mac address output: 
005056c00001
-----------
005056c00001
 - - - - -

device name: 
vmnet8
mac address output: 
005056c00008
-----------
005056c00008
 - - - - -

device name: 
lo
mac address output: 
000000000000
-----------
000000000000
 - - - - -

device name: 
eth0
mac address output: 
002170925bc5
-----------
002170925bc5
 - - - - -
.

Finished in 0.21653 seconds
6 examples, 0 failures

now I have to make it into 2 functions, one the lists the devices, one that lists their mac addresses (see way top of blog)

Iterating through keys of hashmap in Ruby

Way 1:

hashmap.each_key do |key|
  puts key
end

Way 2:

hashmap.each do |elem|
  puts elem[0]
end

todo 6.857

HJ: If you click the function "encrypt message"
You can choose the key to sign it and the key to encrypt it
Maybe we can base on that to add a function that can sign someone's key

Making rspec working on my own computer

Added ethernet.rb  and made rspec work.

To run the rspec, eg:

sudo rspec socket_wrapper_spec.rb

...............

Finished in 7.61 seconds
15 examples, 0 failures

CANNOT be using Wireless connection for the rspec to work. MUST be using Ethernet.
commit c21eb650831d6921e33fa393e5f7d3cd3fdd2d1e

Boring details of how I got here

RawSocket mac should have 6 bytes
Failure/Error: let(:mac) { EtherShell::RawSocket.mac eth_device }
Errno::EPERM:
Operation not permitted - socket(2)

man 2 socket

SOCKET(2)                        Linux Programmer's Manual                       SOCKET(2)

NAME
       socket - create an endpoint for communication

SYNOPSIS
       #include           /* See NOTES */
       #include 

       int socket(int domain, int type, int protocol);

DESCRIPTION
       socket() creates an endpoint for communication and returns a descriptor.

       The  domain  argument  specifies  a communication domain; this selects the protocol
       family which will be  used  for  communication.   These  families  are  defined  in
       <sys/socket.h>.  The currently understood formats include:

       Name                Purpose                          Man page
       AF_UNIX, AF_LOCAL   Local communication              unix(7)
       AF_INET             IPv4 Internet protocols          ip(7)
       AF_INET6            IPv6 Internet protocols          ipv6(7)
       AF_IPX              IPX - Novell protocols
       AF_NETLINK          Kernel user interface device     netlink(7)
       AF_X25              ITU-T X.25 / ISO-8208 protocol   x25(7)
       AF_AX25             Amateur radio AX.25 protocol
       AF_ATMPVC           Access to raw ATM PVCs
       AF_APPLETALK        Appletalk                        ddp(7)
       AF_PACKET           Low level packet interface       packet(7)

..../ether_shell/spec/ether_shell$sudo rspec raw_socket_spec.rb
[sudo] password for haoqili:
.....

Finished in 0.06603 seconds
5 examples, 0 failures

/ethernet/spec$ sudo rspec raw_socket_spec.rb 
[sudo] password for haoqili: 
/usr/lib/ruby/gems/1.8/gems/rspec-core-2.5.1/lib/rspec/core/backward_compatibility.rb:20:in `const_missing': uninitialized constant ShellStub::EtherShell (NameError)
 from /usr/lib/ruby/gems/1.8/gems/rspec-expectations-2.5.0/lib/rspec/expectations/backward_compatibility.rb:6:in `const_missing'
 from /home/haoqili/Desktop/rails/ethernet/spec/support/shell_stub.rb:2
 from /usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb:36:in `gem_original_require'
 from /usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb:36:in `require'
 from /home/haoqili/Desktop/rails/ethernet/spec/spec_helper.rb:8
 from /home/haoqili/Desktop/rails/ethernet/spec/spec_helper.rb:8:in `each'
 from /home/haoqili/Desktop/rails/ethernet/spec/spec_helper.rb:8
 from /usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb:36:in `gem_original_require'
 from /usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb:36:in `require'
 from /home/haoqili/Desktop/rails/ethernet/spec/raw_socket_spec.rb:3
 from /usr/lib/ruby/gems/1.8/gems/rspec-core-2.5.1/lib/rspec/core/configuration.rb:386:in `load'
 from /usr/lib/ruby/gems/1.8/gems/rspec-core-2.5.1/lib/rspec/core/configuration.rb:386:in `load_spec_files'
 from /usr/lib/ruby/gems/1.8/gems/rspec-core-2.5.1/lib/rspec/core/configuration.rb:386:in `map'
 from /usr/lib/ruby/gems/1.8/gems/rspec-core-2.5.1/lib/rspec/core/configuration.rb:386:in `load_spec_files'
 from /usr/lib/ruby/gems/1.8/gems/rspec-core-2.5.1/lib/rspec/core/command_line.rb:18:in `run'
 from /usr/lib/ruby/gems/1.8/gems/rspec-core-2.5.1/lib/rspec/core/runner.rb:55:in `run_in_process'
 from /usr/lib/ruby/gems/1.8/gems/rspec-core-2.5.1/lib/rspec/core/runner.rb:46:in `run'
 from /usr/lib/ruby/gems/1.8/gems/rspec-core-2.5.1/lib/rspec/core/runner.rb:10:in `autorun'
 from /usr/bin/rspec:19

In ./spec/raw_socket_spec.rb, ./spec/socket_wrapper_spec.rb
:%s/EtherShell/Ethernet/g
Didn't fix


After talking to Victor, he told me that I don't have ethernet.rb, the equivalent of ether_shell.rb. So modified ether_shell.rb into ethernet.rb.

Fixed.

Adding SD Card to Android Emulator in Eclipse

1. Navigate to your android-sdk directory's "tools", for example, your equivalent of "android-sdk-linux_x86/tools"
2. Do ./mksdcard 64M YourSdCardName
3a. Run the emulator in Eclipse, do, or if you have Windows, follow this

1. Run > Run Configurations
2. Select the "Target" Tab
3. Scroll down to "Additional Emulator Command Line Options" and add in
4. -sdcard [Your full path to ]/tools/YourSdCardName
5. Apply
6. Run

3b. Run the emulator in the terminal:
        ./emulator -avd YourAndroidEmulatorName -sdcard YourSdCardName

Resizing Android Emulator in Eclipse

If the size of the emulator is too big. You can resize it by:
1. Run > Run Configurations
2. Pick the "Target" tab
3. Scroll down to Additional Emulator Command Line Options and put in: "-scale 0.8" or another number

ZXing

How I set up ZXing

1. download ZXing file
2. Follow the directions in here under "Run" stop at "Run on Android".
   
   A test [URL] can be, in .../zxing-read-only, $ java -cp javase/javase.jar:core/core.jar com.google.zxing.client.j2se.CommandLineRunner http://www.idautomation.com/ucc_ean_128.jpeg
   http://www.idautomation.com/ucc_ean_128.jpeg (format: CODE_128, type: TEXT): Raw result: 81007123452112345678 Parsed result: 81007123452112345678 Also, there were 2 result points. Point 0: (12.5,26.0) Point 1: (167.5,26.0)
3. You should set up a new Android project by doing: File > New > Project > Select "Android Project" under Android > Next > Create project from existing source and navigate to the "android" folder of your downloaded ZXing file. Put a name. and a platform.
4. At this point, I got errors like this:
   
   The import com.google.zxing.BarcodeFormat cannot be resolved (at least in src > com.google.zxing.client.android > CaptureActivity.java)
   Here is the way to fix these errors, adapted from here.
   
   Right click on Project Name
   - Go to New -> Source Folder
   - For Folder name, call it "srccore"
   - Click on Finish
   
   Right click on "srccore"
   - Click on Import
   - Click on File System > Next
   - For "From directory:" type in [full path to]/zxing/core/src
   - Make sure the checkbox next to "src" is checked
   - Finish
   
   Go to Project -> Clean
5. Run it (might take a while), you should see the square moves around:
   
   
   
   
   
   
6. Done! Btw, if you want to resize your emulator.
   

I don't have a real phone, should look at this to fake a camera.

Old notes with things that didn't work

haoqili@stay-focused-hq:13:04:51:~/Desktop/6.857/proj/zxing-read-only$ cd core
haoqili@stay-focused-hq:13:04:52:~/Desktop/6.857/proj/zxing-read-only/core$ ant
Unable to locate tools.jar. Expected to find it in /usr/lib/jvm/java-6-openjdk/lib/tools.jar
Buildfile: /home/haoqili/Desktop/6.857/proj/zxing-read-only/core/build.xml

clean:

build:

init:

compile:
    [mkdir] Created dir: /home/haoqili/Desktop/6.857/proj/zxing-read-only/core/build
    [javac] Compiling 177 source files to /home/haoqili/Desktop/6.857/proj/zxing-read-only/core/build

BUILD FAILED
/home/haoqili/Desktop/6.857/proj/zxing-read-only/core/build.xml:54: The following error occurred while executing this line:
/home/haoqili/Desktop/6.857/proj/zxing-read-only/core/build.xml:37: Error running javac compiler

Total time: 1 second

http://ubuntuforums.org/archive/index.php/t-870670.html


haoqili@stay-focused-hq:13:12:51:~/Desktop/6.857/proj/zxing-read-only/core$ sudo update-alternatives --config java
[sudo] password for haoqili:
There is only one alternative in link group java: /usr/lib/jvm/java-6-openjdk/jre/bin/java
Nothing to configure.


http://www.webupd8.org/2010/10/sun-java-finally-uploaded-to-ubuntu.html

sudo apt-get update

sudo apt-get install sun-java6-jre

sudo apt-get install sun-java6-jdk




haoqili@stay-focused-hq:15:57:05:/usr/lib/jvm$ sudo update-alternatives --config java
There are 2 choices for the alternative java (providing /usr/bin/java).


  Selection    Path                                      Priority   Status
------------------------------------------------------------
* 0            /usr/lib/jvm/java-6-openjdk/jre/bin/java   1061      auto mode
  1            /usr/lib/jvm/java-6-openjdk/jre/bin/java   1061      manual mode
  2            /usr/lib/jvm/java-6-sun/jre/bin/java       63        manual mode


Press enter to keep the current choice[*], or type selection number: 2
update-alternatives: using /usr/lib/jvm/java-6-sun/jre/bin/java to provide /usr/bin/java (java) in manual mode.

haoqili@stay-focused-hq:15:23:34:~/Desktop/6.857/proj/zxing-read-only/core$ sudo update-java-alternatives -l

java-6-openjdk 1061 /usr/lib/jvm/java-6-openjdk

haoqili@stay-focused-hq:15:25:49:~/Desktop/6.857/proj/zxing-read-only/core$ sudo update-java-alternatives -l

[sudo] password for haoqili: 

java-6-openjdk 1061 /usr/lib/jvm/java-6-openjdk

java-6-sun 63 /usr/lib/jvm/java-6-sun

haoqili@stay-focused-hq:15:58:54:~/Desktop/6.857/proj/zxing-read-only/core$ ant

Buildfile: /home/haoqili/Desktop/6.857/proj/zxing-read-only/core/build.xml

clean:

   [delete] Deleting directory /home/haoqili/Desktop/6.857/proj/zxing-read-only/core/build

build:

init:

compile:

    [mkdir] Created dir: /home/haoqili/Desktop/6.857/proj/zxing-read-only/core/build

    [javac] Compiling 177 source files to /home/haoqili/Desktop/6.857/proj/zxing-read-only/core/build

      [jar] Building jar: /home/haoqili/Desktop/6.857/proj/zxing-read-only/core/core.jar

BUILD SUCCESSFUL

Total time: 8 seconds

haoqili@stay-focused-hq:15:59:20:~/Desktop/6.857/proj/zxing-read-only/core$ cd ../javase

haoqili@stay-focused-hq:16:03:32:~/Desktop/6.857/proj/zxing-read-only/javase$ ls

build.xml  pom.xml  src

haoqili@stay-focused-hq:16:03:33:~/Desktop/6.857/proj/zxing-read-only/javase$ ant

Buildfile: /home/haoqili/Desktop/6.857/proj/zxing-read-only/javase/build.xml

init:

build:

    [mkdir] Created dir: /home/haoqili/Desktop/6.857/proj/zxing-read-only/javase/build

    [javac] Compiling 6 source files to /home/haoqili/Desktop/6.857/proj/zxing-read-only/javase/build

      [jar] Building jar: /home/haoqili/Desktop/6.857/proj/zxing-read-only/javase/javase.jar

BUILD SUCCESSFUL

Total time: 6 seconds

haoqili@stay-focused-hq:16:03:44:~/Desktop/6.857/proj/zxing-read-only/javase$ 

haoqili@stay-focused-hq:16:04:27:~/Desktop/6.857/proj/zxing-read-only$ java -cp javase/javase.jar:core/core.jar com.google.zxing.client.j2se.CommandLineRunner [URL]

Exception in thread "main" java.net.URISyntaxException: Illegal character in path at index 0: [URL]

at java.net.URI$Parser.fail(URI.java:2809)

at java.net.URI$Parser.checkChars(URI.java:2982)

at java.net.URI$Parser.parseHierarchical(URI.java:3066)

at java.net.URI$Parser.parse(URI.java:3024)

at java.net.URI.<init>(URI.java:578)

at com.google.zxing.client.j2se.CommandLineRunner.decodeOneArgument(CommandLineRunner.java:210)

at com.google.zxing.client.j2se.CommandLineRunner.main(CommandLineRunner.java:104)

haoqili@stay-focused-hq:16:04:49:~/Desktop/6.857/proj/zxing-read-only$ java -cp javase/javase.jar:core/core.jar com.google.zxing.client.j2se.CommandLineRunner http://www.idautomation.com/ucc_ean_128.jpeg

http://www.idautomation.com/ucc_ean_128.jpeg (format: CODE_128, type: TEXT):

Raw result:

81007123452112345678

Parsed result:

81007123452112345678

Also, there were 2 result points.

  Point 0: (12.5,26.0)

  Point 1: (167.5,26.0)

===========

Android

http://code.google.com/p/zxing/wiki/GetTheReader

http://code.google.com/p/zxing/downloads/detail?name=BarcodeScanner3.53.apk&can=2&q=

http://groups.google.com/group/zxing/browse_thread/thread/7095fb9a0c5c48fe?pli=1 same problem as message one

step 5 of http://knol.google.com/k/alec-go/modifying-the-android-barcode-scanner/2vd5zn6va2fqd/2#

[2011-04-23 16:45:51 - ZXing] ERROR: Unable to open class file /home/haoqili/workspace/ZXing/gen/com/google/zxing/client/android/R.java: No such file or directory

http://stackoverflow.com/questions/3796490/error-unable-to-open-class-file-r-java restarting eclipse fixed that

Then I run emulator, when I click on the scan barcode ap, goes black blank and goes to front page again.

===

Second (Android) part of: 

http://code.google.com/p/zxing/wiki/GettingStarted

http://www.cnblogs.com/sungod/archive/2011/02/28/1967137.html

android-home=/home/haoqili/Desktop/6.857/proj/android-sdk-linux_x86

proguard-jar=/home/haoqili/Desktop/6.857/proj/proguard4.6/lib/proguard.jar

haoqili@stay-focused-hq:17:39:53:~/Desktop/6.857/proj/zxing-read-only/core$ ant clean build-no-debug

Buildfile: /home/haoqili/Desktop/6.857/proj/zxing-read-only/core/build.xml

clean:

   [delete] Deleting directory /home/haoqili/Desktop/6.857/proj/zxing-read-only/core/build

   [delete] Deleting: /home/haoqili/Desktop/6.857/proj/zxing-read-only/core/core.jar

clean:

build-no-debug:

init:

compile:

    [mkdir] Created dir: /home/haoqili/Desktop/6.857/proj/zxing-read-only/core/build

    [javac] Compiling 177 source files to /home/haoqili/Desktop/6.857/proj/zxing-read-only/core/build

      [jar] Building jar: /home/haoqili/Desktop/6.857/proj/zxing-read-only/core/core.jar

BUILD SUCCESSFUL

Total time: 6 seconds

I copied the contents of android to workspace/ZXing, so I went there to "build android"

haoqili@stay-focused-hq:17:41:28:~/workspace/ZXing$ ant

Buildfile: /home/haoqili/workspace/ZXing/build.xml

BUILD FAILED

/home/haoqili/workspace/ZXing/build.xml:64: taskdef class com.android.ant.SetupTask cannot be found

 using the classloader AntClassLoader[]

Total time: 0 seconds

===

Looks like I shouldn't have moved it. 

Eck. Try again by building a project from the file (see top)

The scanner worked [insert pictures]

==
last step of Building android is stil bad

haoqili@stay-focused-hq:22:11:37:~/Desktop/6.857/proj/zxing-read-only/android$ ant
Buildfile: /home/haoqili/Desktop/6.857/proj/zxing-read-only/android/build.xml

BUILD FAILED
/home/haoqili/Desktop/6.857/proj/zxing-read-only/android/build.xml:64: taskdef class com.android.ant.SetupTask cannot be found
 using the classloader AntClassLoader[]

Total time: 0 seconds

system-getifaddrs

The creator of system-getifaddrs, Bruno Coimbra, is so awesome! He helped me fix the problems I had here.

Setting up system-getifaddrs:

Inside the system-getifaddrs directory, I did:

$ sudo gem install system-getifaddrs

[sudo] password for haoqili:
Fetching: system-getifaddrs-0.1.1.gem (100%)
Building native extensions. This could take a while...
Successfully installed system-getifaddrs-0.1.1
1 gem installed
Installing ri documentation for system-getifaddrs-0.1.1...
Installing RDoc documentation for system-getifaddrs-0.1.1...

Then I did:
$ irb
irb(main):001:0> require 'rubygems'
=> true
irb(main):002:0> require 'system/getifaddrs'
=> true
irb(main):003:0> p System.get_ifaddrs

{:wlan0=>{:inet_addr=>"18.111.61.140", :netmask=>"18.111.61.140"}, :vmnet1=>{:inet_addr=>"172.16.76.1", :netmask=>"172.16.76.1"}, :vmnet8=>{:inet_addr=>"172.16.197.1", :netmask=>"172.16.197.1"}, :lo=>{:inet_addr=>"127.0.0.1", :netmask=>"127.0.0.1"}}
=> nil

me: how come eth0 is not in the list?
Bruno: are you connected on wireless?
me: I'm on wireless
Bruno: you are using
           eth0 should be down


Making test.rb work. Add the first line, so it's like:
1 require "rubygems"
2 require "pp"
3 require "system/getifaddrs"
4 pp System.get_ifaddrs

Where pp is for "pretty print".
$ ruby test.rb

{:wlan0=>{:inet_addr=>"18.111.61.140", :netmask=>"18.111.61.140"},
:vmnet1=>{:inet_addr=>"172.16.76.1", :netmask=>"172.16.76.1"},
:vmnet8=>{:inet_addr=>"172.16.197.1", :netmask=>"172.16.197.1"},
:lo=>{:inet_addr=>"127.0.0.1", :netmask=>"127.0.0.1"}}

Without the first line, the result would be as below. I need the "rubygems" because I am running system-wide ruby instead of rvm, changed to it here. (which ruby, gives "/usr/bin/ruby")

test.rb:2:in `require': no such file to load -- system/getifaddrs (LoadError)
from test.rb:2

Where is the *.gem file?
me: is system-getifaddrs a ruby gem?
Bruno: yep
me: how come it doesn't have a *.gem file anywhere?
Bruno: ah, on github have only source files
me: so the *.gem file is not necessary for it to run?
Bruno: *.gem file is the package file like deb packages for debian
             after installing, files are unpacked under ruby lib dir and gem file is discarted
me: so, on github you put the state after unpackaging, that's why .gem doesn't need to be there
Bruno: nope, on github is development sources. with that sources you can packaging its into a gem file

Getting ethernet_ping to work

Goal: recreate scratchpad's ether_ping in my ethernet gem.

AF / PF POSIX Socket API

Ruby learning
ruby array pack() array-->binary string
string unpack string --> array
array first()
attr_reader :to_return is an alias for returning that instance's to_return.

ether_ping  eth0                88B7                       001122334455        aa00112233
ether_ping  net_interface   ether_type 2bytes   dest_mac 6byte        data 0pad to 64 bytes

scratchpad's ether_ping, relies on
- scratchpad/bin/
        - ether_ping is the client
                  client = Scratchpad::Ethernet::PingClient.new interface, ether_type, dest_mac
                  puts client.ping(data)
        - ether_ping_server
                  it seems that calling ether_ping doesn't need ether_ping_server's stuff
- scratchpad/lib/scratchpad/ethernet
        - ping.rb has classes: PingServer and PingClient
                 has require 'eventmachine' for communication
        - raw_ethernet.rb connects to the raw_ethernet stuff

What I did:
1. keep looking at that. Especially socket http://www.ruby-doc.org/stdlib/libdoc/socket/rdoc/classes/Socket.html
2. ethernet is basically ether_shell renamed, with scratchpad's pinging
3. Put in system-getifaddr in too

trying

ethernet$ rake
(in /home/haoqili/Desktop/rails/ethernet)
/usr/bin/ruby1.8 -S bundle exec rspec "spec/raw_socket_spec.rb" "spec/ethernet_spec.rb" "spec/socket_wrapper_spec.rb"
/usr/lib/ruby/gems/1.8/gems/rspec-core-2.3.1/lib/rspec/core/backward_compatibility.rb:20:in `const_missing': uninitialized constant ShellStub::Ethernet (NameError)
from /usr/lib/ruby/gems/1.8/gems/rspec-expectations-2.3.0/lib/rspec/expectations/backward_compatibility.rb:6:in `const_missing'
from /home/haoqili/Desktop/rails/ethernet/spec/support/shell_stub.rb:2
from /home/haoqili/Desktop/rails/ethernet/spec/spec_helper.rb:8:in `require'
from /home/haoqili/Desktop/rails/ethernet/spec/spec_helper.rb:8
from /home/haoqili/Desktop/rails/ethernet/spec/spec_helper.rb:8:in `each'
from /home/haoqili/Desktop/rails/ethernet/spec/spec_helper.rb:8
from /home/haoqili/Desktop/rails/ethernet/spec/raw_socket_spec.rb:4:in `require'
from /home/haoqili/Desktop/rails/ethernet/spec/raw_socket_spec.rb:4
from /usr/lib/ruby/gems/1.8/gems/rspec-core-2.3.1/lib/rspec/core/configuration.rb:388:in `load'
from /usr/lib/ruby/gems/1.8/gems/rspec-core-2.3.1/lib/rspec/core/configuration.rb:388:in `load_spec_files'
from /usr/lib/ruby/gems/1.8/gems/rspec-core-2.3.1/lib/rspec/core/configuration.rb:388:in `map'
from /usr/lib/ruby/gems/1.8/gems/rspec-core-2.3.1/lib/rspec/core/configuration.rb:388:in `load_spec_files'
from /usr/lib/ruby/gems/1.8/gems/rspec-core-2.3.1/lib/rspec/core/command_line.rb:18:in `run'
from /usr/lib/ruby/gems/1.8/gems/rspec-core-2.3.1/lib/rspec/core/runner.rb:55:in `run_in_process'
from /usr/lib/ruby/gems/1.8/gems/rspec-core-2.3.1/lib/rspec/core/runner.rb:46:in `run'
from /usr/lib/ruby/gems/1.8/gems/rspec-core-2.3.1/lib/rspec/core/runner.rb:10:in `autorun'
from /usr/bin/rspec:19
rake aborted!
ruby -S bundle exec rspec "spec/raw_socket_spec.rb" "spec/ethernet_spec.rb" "spec/socket_wrapper_spec.rb" failed

(See full trace by running task with --trace)

ethernet$ sudo rake install
(in /home/haoqili/Desktop/rails/ethernet)
Successfully built RubyGem
Name: ethernet
Version: 0.0.0
File: ethernet-0.0.0.gem
Executing "ruby1.8 -S gem install ./pkg/ethernet-0.0.0.gem":
ruby1.8 -S gem install ./pkg/ethernet-0.0.0.gem
Successfully installed ethernet-0.0.0
1 gem installed
Installing ri documentation for ethernet-0.0.0...
Installing RDoc documentation for ethernet-0.0.0...


========
added "s.executables =["ethernet_ping"]" in ethernet.gemspec

/ethernet$ sudo rake install
[sudo] password for haoqili:
(in /home/haoqili/Desktop/rails/ethernet)
Successfully built RubyGem
Name: ethernet
Version: 0.0.0
File: ethernet-0.0.0.gem
Executing "ruby1.8 -S gem install ./pkg/ethernet-0.0.0.gem":
ruby1.8 -S gem install ./pkg/ethernet-0.0.0.gem
Building native extensions. This could take a while...
Successfully installed eventmachine-0.12.10
Successfully installed ethernet-0.0.0
2 gems installed
Installing ri documentation for eventmachine-0.12.10...
Installing ri documentation for ethernet-0.0.0...
Installing RDoc documentation for eventmachine-0.12.10...
Could not find main page README
Could not find main page README
Could not find main page README
Could not find main page README
Installing RDoc documentation for ethernet-0.0.0...


still not found
ethernet$ ethernet_ping
ethernet_ping: command not found

deleted s.executables = ["ethernet_ping"] from ethernet.gemspec
added gem.executables = ["ethernet_ping"] to Rakefile
$ sudo rake install
[sudo] password for haoqili:
(in /home/haoqili/Desktop/rails/ethernet)
Successfully built RubyGem
Name: ethernet
Version: 0.0.0
File: ethernet-0.0.0.gem
Executing "ruby1.8 -S gem install ./pkg/ethernet-0.0.0.gem":
ruby1.8 -S gem install ./pkg/ethernet-0.0.0.gem
Building native extensions. This could take a while...
Successfully installed eventmachine-0.12.10
Successfully installed ethernet-0.0.0
2 gems installed
Installing ri documentation for eventmachine-0.12.10...
Installing ri documentation for ethernet-0.0.0...
Installing RDoc documentation for eventmachine-0.12.10...
Could not find main page README
Could not find main page README
Could not find main page README
Could not find main page README
Installing RDoc documentation for ethernet-0.0.0...

haoqili@stay-focused-hq:18:32:11:~/Desktop/rails/ethernet$ ethernet_ping
Usage: /usr/bin/ethernet_ping net_interface ether_type dest_mac data
net_interface: name of the Ethernet interface, e.g. eth0
ether_type: packet type for the Ethernet II frame, in hex (2 bytes)
dest_mac: destination MAC for the ping packets, in hex (6 bytes)
data: ping packet data, in hex (0-padded to 64 bytes)

haoqili@stay-focused-hq:18:35:52:~/Desktop/rails/ethernet$ sudo ethernet_ping eth0 88B7 001122334455 aa00112233
/usr/lib/ruby/gems/1.8/gems/ethernet-0.0.0/bin/ethernet_ping:22: uninitialized constant Ethernet (NameError)
from /usr/bin/ethernet_ping:19:in `load'
from /usr/bin/ethernet_ping:19

19 load Gem.bin_path('ethernet', 'ethernet_ping', version)

That doesn't look right

* After Victor told me to add ethernet.rb (and then I did sudo rake install), the Ethernet NameError is gone, but now I have

$ sudo ethernet_ping eth0 88B7 001122334455 aa00112233
[sudo] password for haoqili:
/usr/lib/ruby/gems/1.8/gems/ethernet-0.0.0/bin/ethernet_ping:22: uninitialized constant Ethernet::PingClient (NameError)
from /usr/bin/ethernet_ping:19:in `load'
from /usr/bin/ethernet_ping:19

* Ah, because I never put in the file about ethernet pinging in "ethernet.rb", so I added this line:
require 'ping.rb' # This is to get ethernet_ping working :P

sudo rake install again
$ sudo ethernet_ping eth0 88B7 001122334455 aa00112233
/usr/lib/ruby/gems/1.8/gems/ethernet-0.0.0/lib/ping.rb:63:in `initialize': undefined method `socket' for Ethernet:Module (NoMethodError)
from /usr/lib/ruby/gems/1.8/gems/ethernet-0.0.0/bin/ethernet_ping:22:in `new'
from /usr/lib/ruby/gems/1.8/gems/ethernet-0.0.0/bin/ethernet_ping:22
from /usr/bin/ethernet_ping:19:in `load'
from /usr/bin/ethernet_ping:19

This is because I put my "socket" method inside the RawSocket module as well, so for methods ".socket" and ".mac", I have to change their prefix from "Ethernet::" to "Ethernet::RawSocket".


I also reordered where the ruby files are stored. Victor told me that the way he set ether_shell up is how it's done by convention. "IDEs have shortcuts that let you navigate between code and spec, as long as things stay where they're supposed to be."

so after "sudo rake install", I tried the above ethernet_ping, but I got another error:

$ sudo ethernet_ping eth0 88B7 001122334455 aa00112233
/usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb:36:in `gem_original_require': no such file to load -- ethernet/raw_socket.rb (LoadError)
from /usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb:36:in `require'
from /usr/lib/ruby/gems/1.8/gems/ethernet-0.0.0/lib/ethernet.rb:6
from /usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb:36:in `gem_original_require'
from /usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb:36:in `require'
from /usr/lib/ruby/gems/1.8/gems/ethernet-0.0.0/bin/ethernet_ping:3
from /usr/bin/ethernet_ping:19:in `load'
from /usr/bin/ethernet_ping:19

This is at commit 1b6daf320e82219a15d52d734dcbb10aff1b5fb6.

The next day, I went to see Victor. He told me to check:

$ ls /usr/lib/ruby/gems/1.8/gems/ethernet-0.0.0/lib
ethernet.rb

Indeed, the ethernet dir was not there.

I did again:

ethernet$ sudo rake install
(in /home/haoqili/Desktop/rails/ethernet)
  Successfully built RubyGem
  Name: ethernet
  Version: 0.0.0
  File: ethernet-0.0.0.gem
Executing "ruby1.8 -S gem install ./pkg/ethernet-0.0.0.gem":
ruby1.8 -S gem install ./pkg/ethernet-0.0.0.gem
Building native extensions.  This could take a while...
Successfully installed eventmachine-0.12.10
Successfully installed ethernet-0.0.0
2 gems installed
Installing ri documentation for eventmachine-0.12.10...
Installing ri documentation for ethernet-0.0.0...
Installing RDoc documentation for eventmachine-0.12.10...
Could not find main page README
Could not find main page README
Could not find main page README
Could not find main page README

$ ls /usr/lib/ruby/gems/1.8/gems/ethernet-0.0.0/lib
ethernet  ethernet.rb
haoqili@stay-focused-hq:15:16:56:~/Desktop/rails/ethernet$ sudo ethernet_ping eth0 88B7 001122334455 aa00112233
Pinging 001122334455... 

Fixed!

Shoot. It didn't show up on Wireshark :/

I printed out the source mac, sure it doesn't match eth0's mac address:
eth0 Link encap:Ethernet HWaddr 00:21:70:92:5b:c5

57 class PingClient
 58   def initialize(if_name, ether_type, destination_mac)
 59     @socket = Ethernet.socket if_name, ether_type
 60 
 61     @source_mac = [Ethernet.get_interface_mac(if_name)].pack('H*')[0, 6]
 62     puts "\n the source_mac, is"
 63     puts @source_mac.unpack('H*')[0]

ethernet's result:

$ sudo ethernet_ping eth0 88B7 001122334455 aa00112233

 the source mac is:
0192b5
Pinging 001122334455... 

scratchpad's result: (scratchpad has the working ether_ping)

$ sudo ether_ping eth0 88B7 001122334455 aa00112233

 the source_mac, is
002170925bc5
Pinging 001122334455... 

This is because ether_shell's raw_socket's .mac() doesn't have "unpack('H*').first", so if I put this back in, pinging would work in ethernet

raw_socket.rb
 32   def self.mac(eth_device)
 33     case RUBY_PLATFORM
 34     when /linux/
 35       # /usr/include/net/if.h, structure ifreq
 36       ifreq = [eth_device].pack 'a32'
 37       # 0x8927 is SIOCGIFHWADDR in /usr/include/bits/ioctls.h
 38       socket.ioctl 0x8927, ifreq
 40       #ifreq[18, 6] #in scratchpad's raw_ethernet, then says .unpack('H*').first
 41                    # because raw socket should return raw data
 42                    # leaving higher layers for presentation issues
 43     else

add it in here instead

ping.rb
66     @source_mac = [Ethernet::RawSocket.mac(eth_device).unpack('H*').first].pack('H*')[0, 6]

Yay worked!

commit: bff1c0ca79ed679aae77e5e6199c2aaf3e75fdc9

:) ethernet_ping works! Try with 'sudo ethernet_ping eth0 88B7 001122334455 aa00112233' and see it in Wireshark's eth0

How Wireshark lists devices

Goal: find how Wireshark gets the list of MAC addresses at the beginning (Windows, MacOS)
Edit: Later found out that neither Wireshark nor libpcap gives a list of MAC addresses.
Answer: To get a list of internet device names, Wireless uses dumpcap's pcap_findalldevs().

I. I looked for MAC address
II. I looked for just the simple list of devices Wireshark shows

I. I grep-ed for "MAC address" and found over 500 entries, under these big categories：
- epan/dissectors (huge)
- plugins
- airpcap

Interesting things found:

• Ulf Lamping, ulf.lamping[at]web.de, put the source and destination MAC addresses into the top-level item for Ethernet.
• "manuf" contains Ethernet vendor codes (i.e. the first half of MAC Addresses), and well-known MAC addresses. So I checked the MAC Addresses from my ifconfig -a. eth0's first half is 00:21:70, which manuf says is Dell (yes I have a Dell computer). vmnet*'s first half is 00:50:56, which manuf says is VMware. wlan0's is 00:1c:26, which manuf says is Hon Hai Precision Ind. Co. Cool.
  
• Looking at epan/dissectors doesn't seem right. Hmm, 
• Talked to Victor about it, he told me to look at the RubyGem (see other post) first

II. Give up. So what if I just track it down? The list at the beginning of Wireshark appears below the words "Start capture on interface:". So I grep-ed for those words and found that it appears in ./gtk/main_welcome.c.

There are 2 ways to go in main_welcome.c
1. WIRESHARK_STOCK_CAPTURE_INTERFACES from clicking on "Interface List" button on the main page. This is from

812 welcome_button(WIRESHARK_STOCK_CAPTURE_INTERFACES,
813         "Interface List",
814         "Live list of the capture interfaces\n(counts incoming packets)", ...

But searching down this path wasn't very fruitful

2a. By looking at how the list of devices is generated directly with the welcome_if_panel_load() function.

653 /* load the list of interfaces */
654 static void
655 welcome_if_panel_load(void)

661     GList         *if_list;
662     int err;
663     gchar         *err_str = NULL;


669     /* LOAD THE INTERFACES */

670     if_list = capture_interface_list(&err, &err_str);

btw, the g_blah() functions come from GLib

2b. So I now grep-ed for capture_interface_list(), which attempts to open all adapters it finds in order to check whether they can be captured on.

capture_interface_list() in ./capture_ifinfo.c

 1 /* capture_ifinfo.c
  2  * Routines for getting interface information from dumpcap
 23  */
 24
 25 #ifdef HAVE_CONFIG_H
 26 # include "config.h"
 27 #endif
 28
 29 #ifdef HAVE_LIBPCAP
 30
 31 #include <stdlib.h>
 32 #include <string.h>
 33
 34 #ifdef HAVE_ARPA_INET_H
 35 #include <arpa/inet.h>
 36 #endif
 37
 38 #ifdef HAVE_SYS_SOCKET_H
 39 #include <sys/socket.h>         /* needed to define AF_ values on UNIX */
 40 #endif
 41
 42 #ifdef HAVE_WINSOCK2_H
 43 #include <winsock2.h>           /* needed to define AF_ values on Windows */

 44 #endif

 45 

 46 #ifdef NEED_INET_V6DEFS_H

 47 # include "wsutil/inet_v6defs.h"

 48 #endif

 49 

 50 #include <glib.h>

 51 

 52 #include "capture_opts.h"

 53 #include "capture_sync.h"

 54 #include "log.h"

 55 

 56 #include "capture_ifinfo.h"

 57 

 58 /**

 59  * Fetch the interface list from a child process (dumpcap).

 60  *

 61  * @return A GList containing if_info_t structs if successful, NULL (with err and possibly err_str set) otherwise.

 62  *

 63  */

 64 

 65 /* XXX - We parse simple text output to get our interface list.  Should

 66  * we use "real" data serialization instead, e.g. via XML? */

 67 GList *

 68 capture_interface_list(int *err, char **err_str)

 69 {

 70     int        ret;

 71     GList     *if_list = NULL;

 72     int        i, j;

 73     gchar     *data, *primary_msg, *secondary_msg;

 74     gchar    **raw_list, **if_parts, **addr_parts;

 75     gchar     *name;

 76     if_info_t *if_info;

 77     if_addr_t *if_addr;

 78 

 79     g_log(LOG_DOMAIN_CAPTURE, G_LOG_LEVEL_MESSAGE, "Capture Interface List ...");

 80 

 81     /* Try to get our interface list */

 82     ret = sync_interface_list_open(&data, &primary_msg, &secondary_msg);

 83     if (ret != 0) {

 84         g_log(LOG_DOMAIN_CAPTURE, G_LOG_LEVEL_MESSAGE, "Capture Interface List failed!");

 85         if (err_str) {

 86             *err_str = primary_msg;

 87         } else {

 88             g_free(primary_msg);

 89         }

 90         g_free(secondary_msg);

 91         *err = CANT_GET_INTERFACE_LIST;

 92         return NULL;

 93     }

 94 

 95     /* Split our lines */

 96 #ifdef _WIN32

 97     raw_list = g_strsplit(data, "\r\n", 0);

 98 #else

 99     raw_list = g_strsplit(data, "\n", 0);

100 #endif

101     g_free(data);

102 

103     for (i = 0; raw_list[i] != NULL; i++) {

104         if_parts = g_strsplit(raw_list[i], "\t", 4);

105         if (if_parts[0] == NULL || if_parts[1] == NULL || if_parts[2] == NULL ||

106                 if_parts[3] == NULL) {

107             g_strfreev(if_parts);

108             continue;

109         }

110 

111         /* Number followed by the name, e.g "1. eth0" */

112         name = strchr(if_parts[0], ' '); /*strchr searches to see if there is a single
                                                                space chr in if_parts[0] */

113         if (name) {

114             name++;

115         } else {

116             g_strfreev(if_parts);

117             continue;

118         }

119 

120         if_info = g_malloc0(sizeof(if_info_t));

121         if_info->name = g_strdup(name);

122         if (strlen(if_parts[1]) > 0)

123             if_info->description = g_strdup(if_parts[1]);

124         addr_parts = g_strsplit(if_parts[2], ",", 0);

125         for (j = 0; addr_parts[j] != NULL; j++) {

126             if_addr = g_malloc0(sizeof(if_addr_t));

127             if (inet_pton(AF_INET, addr_parts[j], &if_addr->addr.ip4_addr)) {

128                 if_addr->ifat_type = IF_AT_IPv4;

129             } else if (inet_pton(AF_INET6, addr_parts[j],

130                     &if_addr->addr.ip6_addr)) {

131                 if_addr->ifat_type = IF_AT_IPv6;

132             } else {

133                 g_free(if_addr);

134                 if_addr = NULL;

135             }

136             if (if_addr) {

137                 if_info->addrs = g_slist_append(if_info->addrs, if_addr);

138             }

139         }

140         if (strcmp(if_parts[3], "loopback") == 0)

141             if_info->loopback = TRUE;

142         g_strfreev(if_parts);

143         g_strfreev(addr_parts);

144         if_list = g_list_append(if_list, if_info);

145     }

146     g_strfreev(raw_list);

147 

148     /* Check to see if we built a list */

149     if (if_list == NULL) {

150         *err = NO_INTERFACES_FOUND;

151         if (err_str)

152             *err_str = g_strdup("No interfaces found");

153     }

154     return if_list;

155 }

2c. Then look at: sync_interface_list_open(), which is found in ./capture_sync.c:

1073 /*
1074  * Get the list of interfaces using dumpcap.
1075  *
1076  * On success, *data points to a buffer containing the dumpcap output,
1077  * *primary_msg and *secondary_msg are NULL, and 0 is returned.  *data
1078  * must be freed with g_free().
1079  *
1080  * On failure, *data is NULL, *primary_msg points to an error message,
1081  * *secondary_msg either points to an additional error message or is
1082  * NULL, and -1 is returned; *primary_msg, and *secondary_msg if not NULL,
1083  * must be freed with g_free().
1084  */
1085 int
1086 sync_interface_list_open(gchar **data, gchar **primary_msg,
1087                          gchar **secondary_msg)
1088 {
1089     int argc;
1090     const char **argv;
1091
1092     g_log(LOG_DOMAIN_CAPTURE, G_LOG_LEVEL_DEBUG, "sync_interface_list_open");
1093
1094     argv = init_pipe_args(&argc);
1095
1096     if (!argv) {
1097         *primary_msg = g_strdup("We don't know where to find dumpcap.");
1098         *secondary_msg = NULL;
1099         *data = NULL;
1100         return -1;
1101     }
1102
1103     /* Ask for the interface list */
1104     argv = sync_pipe_add_arg(argv, &argc, "-D");
1105
1106 #ifndef DEBUG_CHILD
1107     /* Run dumpcap in capture child mode */
1108     argv = sync_pipe_add_arg(argv, &argc, "-Z");
1109     argv = sync_pipe_add_arg(argv, &argc, SIGNAL_PIPE_CTRL_ID_NONE);
1110 #endif
1111     return sync_pipe_run_command(argv, data, primary_msg, secondary_msg);
1112 }

It's passing in an argument to get the devices, and the argument is "dumpcap"

2d. dumpcap and dumpcap.c
dumpcap is cross platform. :)
man dumpcap told me that the option "-D" prints a list of the interfaces on which Dumpcap can capture.

I found dumpcap's code in "dumpcap.c":

3792      * "-D" requires no interface to be selected; it's supposed to list
3793      * all interfaces.
3794      */
3795     if (list_interfaces) {
3796         /* Get the list of interfaces */
3797         GList       *if_list;
3798         int         err;
3799         gchar       *err_str;
3800 
3801         if_list = capture_interface_list(&err, &err_str);

So then I track down this function

796 GList *
 797 capture_interface_list(int *err, char **err_str)
 798 {
 799     return get_interface_list(err, err_str);
 800 }

after "grep -ir get_interface_list ." I found it in:

2e. capture-wpcap.c

676 /*
677  * This will use "pcap_findalldevs()" if we have it, otherwise it'll
678  * fall back on "pcap_lookupdev()".
679  */
680 GList *
681 get_interface_list(int *err, char **err_str)
682 {
683     GList  *il = NULL;
684     wchar_t *names;
685     char *win95names;
686     char ascii_name[MAX_WIN_IF_NAME_LEN + 1];
687     char ascii_desc[MAX_WIN_IF_NAME_LEN + 1];
688     int i, j;
689     char errbuf[PCAP_ERRBUF_SIZE];
690 
691 #ifdef HAVE_PCAP_FINDALLDEVS
692     if (p_pcap_findalldevs != NULL)
693         return get_interface_list_findalldevs(err, err_str);
694 #endif
...
738 
739     names = (wchar_t *)pcap_lookupdev(errbuf);

There are 2 methods, get_interface_list_findalldevs() or pcap_lookupdev()

Both methods will require code from libpcap, which I found on this site.

Working out the 2 different ways

2d.1. First, let's try with get_interface_list_findalldevs()
Inside capture-pcap-util.c

107 #ifdef HAVE_PCAP_FINDALLDEVS
...
166 GList *
167 get_interface_list_findalldevs(int *err, char **err_str)
168 {
169     GList  *il = NULL;
170     pcap_if_t *alldevs, *dev;
171     if_info_t *if_info;
172     char errbuf[PCAP_ERRBUF_SIZE];
173 
174     if (pcap_findalldevs(&alldevs, errbuf) == -1) {
175         *err = CANT_GET_INTERFACE_LIST;
176         if (err_str != NULL)
177             *err_str = cant_get_if_list_error_message(errbuf);
178         return NULL;
179     }
180 
181     if (alldevs == NULL) {
182         /*
183          * No interfaces found.
184          */
185         *err = NO_INTERFACES_FOUND;
186         if (err_str != NULL)
187             *err_str = NULL;
188         return NULL;
189     }
190 
191     for (dev = alldevs; dev != NULL; dev = dev->next) {
192         if_info = if_info_new(dev->name, dev->description);
193         il = g_list_append(il, if_info);
194         if_info_ip(if_info, dev);
195     }
196     pcap_freealldevs(alldevs);
197 
198     return il;
199 }
200 #endif /* HAVE_PCAP_FINDALLDEVS */

2d.2 Now, let's try with the other method pcap_lookupdev()

233 char*
234 pcap_lookupdev (char *a)
235 {
236     if (!has_wpcap) {
237         return NULL;
238     }
239     return p_pcap_lookupdev(a);
240 }

which is a pointer

56 static char*   (*p_pcap_lookupdev) (char *);

2d.2b. Now we have to look at libpcap's code for pcap_lookupdev, which will also lead to

pcap_findalldevs()

Found it in inet.c

635 #if !defined(WIN32) && !defined(MSDOS)
636 
637 /*
638  * Return the name of a network interface attached to the system, or NULL
639  * if none can be found.  The interface must be configured up; the
640  * lowest unit number is preferred; loopback is ignored.
641  */
642 char *
643 pcap_lookupdev(errbuf)
644     register char *errbuf;
645 {
...
654     if (pcap_findalldevs(&alldevs, errbuf) == -1)

777 #elif defined(WIN32)
778 
779 /*
780  * Return the name of a network interface attached to the system, or NULL
781  * if none can be found.  The interface must be configured up; the
782  * lowest unit number is preferred; loopback is ignored.
783  */
784 char *
785 pcap_lookupdev(errbuf)
786     register char *errbuf;
787 {
788     DWORD dwVersion;
789     DWORD dwWindowsMajorVersion;
790     dwVersion = GetVersion();   /* get the OS version */
791     dwWindowsMajorVersion = (DWORD)(LOBYTE(LOWORD(dwVersion)));
792 
793     if (dwVersion >= 0x80000000 && dwWindowsMajorVersion >= 4) {
794         /*
795          * Windows 95, 98, ME.
796          */
797         ULONG NameLength = 8192;
798         static char AdaptersName[8192];
799 
800         if (PacketGetAdapterNames(AdaptersName,&NameLength) )
801             return (AdaptersName);
802         else
803             return NULL;
804     } else {
805         /*
806          * Windows NT (NT 4.0, W2K, WXP). Convert the names to UNICODE for backward compatibiliy

3. Now we need to look inside libpcap-1.1.1 for pcap_findalldevs()

Method description from pcap-sita.html:
SMP The Supervisory Management Processor where Wireshark (or equivalent) runs in conjuction with a libpcap front-end.
IOP I/O Processors where the monitored ports exist in conjunction with a custom device driver/libpcap back-end.

pcap_findalldevs constructs a list of network devices that can be opened with pcap_open_live().

SMP It obtains a list of IOPs currently available (via /etc/hosts).
SMP -> IOP The SMP will sequentially open a connection to each IOP on its 'sniffer' port to ensure the IOP is available. It sends a null terminated empty interface ID followed by the query request command.
IOP -> SMP The IOP returns an error response and its list of devices.
SMP -> IOP The SMP closes the TCP connection with each IOP.
SMP The SMP adds the received information to its internal structure.

=========================
SMP/IOP Inter-Process Communication Protocol

Communications between an SMP and an IOP consists of a TCP session
between an ephemeral port on the SMP and the well known port of 49152
(which is the first available port in the 'dynamic and/or private port'
range) on an IOP.

Following a TCP open operation the IOP receives a null terminated
'interface ID' string to determine the type of operation that follows:

Every command received by an IOP implies a 'stop trace/stop forwarding' operation must
occur before executing the received command.

A session is closed when the SMP closes the TCP session with the IOP.
Obviously monitoring and forwarding is also stopped at that time.

Note: All multi-octet entities are sent in network neutral order.

SMP -> IOP	Open socket (to each IOP), and sends: Name/ Purpose Size (in bytes) Description Interface ID 1 A NULL to indicate an an empty 'interface ID'.
IOP -> SMP	Send its (possibly empty) NULL terminated error response string.
SMP -> IOP	Sends the 'interface query request': Name/ Purpose Size (in bytes) Description Interface ID 1 A 'Q' (indicating 'interface query request').
IOP -> SMP	The IOP returns a list of sequences of information as defined by the return parameter of this function call (as shown in the following table). Elements are specified by providing an unsigned byte preceeding the actual data that contains length information. Notes: Name/ Purpose Size (in bytes) Description length 1 The number of octets in the name field that follows. Name 1-255 The name of the interface. The format of the name is an alphabetic string (indicating the type of interface) followed by an optional numeric string (indicating the interface's sequence number). Sequence numbers (if needed) will begin at zero and progress monotonically upwards. (i.e. 'eth0', 'lo', 'wan0', etc.) For an IOP, the alphabetic string will be one of: 'eth', 'wan', and 'lo' for Ethernet, WAN ports and the IP loopback device respectively. An IOP currently supports: 'eth0', 'eth1', 'lo', 'wan0' ... 'wan7'. Note: IOPs and ACNs will not currently support the concept of 'any' interface. length 1 The number of octets in the interface description field that follows. Interface Description 0-255 A description of the interface or it may be an empty string. (i.e. 'ALC') Interface Type 4 The type of interface as defined in the description for pcap_datalink() (in network neutral order). Loopback Flag 1 1 = if the interface is a loopback interface, zero = otherwise. count 1 # of address entries that follow. Each entry is a series of bytes in network neutral order. See the parameter definition above for more details. Repeated 'count' number of times. length 1 The number of octets in the address field that follows. Address 1-255 The address of this interface (in network neutral order). length 1 The number of octets in the netmask field that follows. Network Mask 0-255 The network mask used on this interface (if applicable) (in network neutral order). length 1 The number of octets in the broadcast address field that follows. Broadcast Address 0-255 The broadcast address of this interface (if applicable) (in network neutral order). length 1 The number of octets in the destination address field that follows. Destination Address 0-255 The destination address of this interface (if applicable) (in network neutral order).
SMP -> IOP	Close the socket.
IOP -> SMP	Close the socket.

=========================


I have "pcap_findalldevs" source from the following files in libpcap-1.1.1:
they all take the same arguments
fad-win32.c - for Windows OS

216  * Win32 implementation, based on WinPcap
217  */
218 int
219 pcap_findalldevs(pcap_if_t **alldevsp, char *errbuf)

pcap-dos.c - for DOS (predecessor of Windows)
fad-glif.c - for Solaris
fad-gifc.c - This is the implementation used on platforms that have SIOCGIFCONF but don't have any other mechanism for getting a list of interfaces.
fad-getad.c - This is the implementation used on platforms that have "getifaddrs()"

(btw MacOS uses free bsd.)

Already have a gem to get Linux list MAC Address working
MacOS can be on hold
Focus on getting Windows to work! look at fad-win32.c in more detail

MacOS MAC Address

I emailed Victor with this link that gets MAC addresses on MacOS. His reply:


That is one way of going about it, but it's probably very painful. You'd have to make libffi bindings for all those OSX-specific calls.

Google took me here: http://forum.soft32.com/mac/sample-code-MAC-IP-address-Mac-OS-ftopict45416.html

Then I saw the comment at the bottom, and it makes some good points. I googled for a ruby gem that wraps getifaddres, and found this. https://github.com/bbcoimbra/system-getifaddrs

See if you can use it to get the network card information. If it works, you'll probably have to figure out how to build Windows-specific gems, because I'm almost sure this won't install on Windows.

===

So I git cloned "system-getifaddrs".

The problems are fixed here.

Problem 1: I can't the test.rb from its README to work

I found a extconf.rb in system-getifaddrs/ext/rb_getifaddrs, and did:

$ ruby extconf.rb
checking for arpa/inet.h... yes
checking for sys/socket.h... yes
checking for sys/types.h... yes
checking for netdb.h... yes
checking for ifaddrs.h... yes
checking for stdio.h... yes
checking for stdlib.h... yes
checking for unistd.h... yes
creating Makefile

I don't think that's what I'm supposed to do
Problem 2: How do I up "system-getifaddrs" to use it?

Problem 3: If system-getifaddrs is a ruby gem, where is the ".gem" file?

Reading Wireshark

Goal: find how Wireshark gets the MAC address on Windows and MacOS

To reach the goal:
1. finish reading Wireshark docs, see notes below
2. read victor email, see this post
3. read grep of "mac address" in doc. It was useless
4. read grep of "mac address" from wireshark root
5. email wireshark dev
6. How to get Linux MAC address? look in ether_shell

At first, I grep-ed for "MAC address" in the entire Wireshark directory with grep -ir "mac address" . > hq_macaddsearch.txt. It took way to long to complete so I stopped it. Then I looked that it already had over 150,000 lines where the bottom is filled with "hq_macaddsearch.txt" ... oh I made the infinite loop mistake again, should have saved it to ../hq_macaddsearch.txt :/

I grep-ed for "Mac address" in the entire Wireshark directory with grep -ir "mac address" . and found over 500 entries. I would go back to it after reading the docs to understand which entries are important.

Reading Wireshark docs:

• The overview of the Wireshark
  
  • Dumpcap needs elevated privileges to do the capturing. It has code customized for each OS. It talks with WinPcap or libpcap, platform-dependent packet capture libraries to capture from different types of interfaces.
    
• In the different dissectors in epan/dissectors decodes a layer of the protocol before handing it to the next level of dissector.
• Wireshark is based on the GTK toolkit for the UI.
• Not very helpful in trying to find MAC addresses

epan = Ethereal Packet ANalyzer
ett = Ethereal Tree Type
hf = header field
gchar* = pointer to character string
guint8 = 1-byte value
guint16 = 2-byte value
tvb = Testy Virtual Buffer

Reading Wireshark & Ethereal Network Protocol Analyzer Toolkit

• every NIC (Network Interface Card) has a unique MAC Address
• MAC adress is a 48-bit number, aka 12-digit hexadecimal number. The first half identifies the manufacturer of the Ethernet card, last 24 bits is a serial number assigned. e.g.:12:34:56:78:9a:bc
• View MAC Address on different operating systems:
• Windows XP: command line type ipconfig /all, under Physical Address
  
• Linux: command line type ifconfig -a, listed as "HWaddr". "lo" doesn't have a MAC address, because it's a loop back address only. On other computersarp -a might be useful too
  
• MacOS: command line type ifconfig -a, listed as "Ether" label.

Vimperator!

I like Vim and Firefox, so I got Vimperator today.

Looking for things in Linux

Search for part of a filename:

find . -name "*partOfFileName*"

Search for part of content:

grep -ir stringToSearch .

Search for previous command:

history | grep partOfCommand

merge pdf's on linux

Use the command PDFTK!!

pdftk *.pdf cat output result.pdf

pdftk first.pdf second.pdf third.pdf cat output result.pdf

it does splits too!

Setting up Android Privacy Guard + Bouncy Castle from scratch

The APG project suggested using the Bouncy Castle to as a way to learn how to building the APG. Here are some very detailed steps on how to do this from complete scratch, as if it's your first Android project.

The order of getting things working will be:
I. Set up Eclipse with Android Development SDK and ADT emulator
II. Create a new Android Project with APG and Bouncy Castle incoporated in

I. Setting up Android Development on Eclipse, roughly following the Android SDK Installation Guide.

1. Download Eclipse Classic. (Windows: if it asks, install Java SDK)
   
2. Download the Android SDK. (Windows: if it says Java SDK not found when you have installed it, hit "Back" and "Next" :/ ... good job Windows)
   
3. Get the Eclipse ADT Plugin for Android Emulator in Eclipse by doing:
   Select Help > Install New Software. Add at top right, Name: "ADT Plugin", Location: "https://dl-ssl.google.com/android/eclipse/", or "http: ..". Okay. Next. Next. Accept. Finish. Restart Now.
   
Check that this exists: "Window > Android SDK and AVD Manager" If you get this error: "Location of the Android SDK has not been setup in the preferences". You need to do: a) Window > Preferences > Android b) Put in appropriate path file for the "android-sdk-[OS name]" directory in SDK Location, which should come from your "Android SDK" download (If you downloaded the .exe for Windows, download the zip) c) Apply. OK
4. Install Android Packages on ADT:
   Click on "Window > Android SDK and AVD Manager"
   Click on "Available packages" on the left.
   Check the "Android Repository".
   Press "Install Selected" and "Install"
   

II. Creating our own Android Virtual Device (AVD) with the APG Project and Bouncy Castle installed on it, roughly following the Android Creating AVDs Guide.

1. If you don't have SVN, install it. (For Windows, download TortoiseSVN will require restart)
2. SVN checkout the APG source code. (If you're using TortoiseSVN. Create a folder named "android-privacy-guard-read-only", right click on it, click on "SVN Checkout", and paste the url in.)
3. Download the Bouncy Castle source code, which should extract to "bc145_reduced_for_apg" folder
4. Create a new Android project on Eclipse:
   - File > New > Project
   - Click on Android > Android Project. Next. Fill in the rest as the picture shows:
   
5. Replace with the APG source code: Navigate to workspace/Apg and replace everything inside there with the files in the Apg source you downloaded in the previous step (so replace src, res, AndroidManifest.xml, etc). 
6. Add the Bouncy Castle code: copy the bouncycastle2 directory (inside bc145_reduced_for_apg/org) into workspace/Apg/src/org
7. Run > run: (warning, it'll take a while, a black blank screen doesn't mean it's broken). You should see something like:
   
   
8. Extra: If the size of the emulator is too big. You can resize it by:
   Run > Run Configurations > Target [a tab]
   Write in Additional Emulator Command Line Options: "-scale 0.8" or another number
9. DONE!

Useless stuff: the long way of doing #4 onwards

1. Create a new Android project on Eclipse:
   - File > New > Project
   - Click on Android > Android Project. Next.
   - Project name: "6857proj". Target Name: "Android 1.6". Package Name: "org.thialfihar.android.apg". Create Activity: "proj6857". Finish.
   
2. Make a new AVD for this project:
   - select "6857proj" form the "Package Explorer" on the left side.
   - Run > Run
   - "Yes" to "No compatible targets were found. Do you wish to add a new AVD?"
   - "New" on top right
   - Name: "6857avd" Target: "Android 1.6". Create AVD
   - Select on the "6857avd" and click "Start". "Launch".
   - Wait for 3+ minutes for the Android emulator to turn on
   
3. Add the APG source code into our Android Project:
   - copy all the files and folders inside "android-privacy-guard-read-only -> src -> org -> thialfihar -> android -> apg" into to "[your Eclipse] workspace -> 6857proj -> src -> org -> thialfihar -> android -> apg"
   
4. Add the Bouncy Castle code into our Android Project:
   - copy this directory "bc145_reduced_for_apg -> org -> bouncycastle2" into "[your Eclipse] workspace -> 6857proj -> src -> org". So you should end up with "thialfihar" and "bouncycastle2" in the "org" directory.
   
5. Put the APG resources into the res of the Android Project:
   - delete the "res" directory in the workspace, and replace it with APG's "res"
   
6. Run > run: (warning, it'll take a while, a black blank screen doesn't mean it's broken)
   If you can't run due to errors like "The method xx of type xx must override a superclass", do:
   - Project > Properties
   - Under "Java Compiler," change the compiler compliance level from "1.5" to "1.6". Apply.
   - Now you should be able to run.
   
7. Done!
   
8. To fix this warning below, you have to replace the "AndroidManifest.xml" with the Apg's version too.
   
   Sorry!
   The application APG (process org.thialfihar.android.apg) has stopped unexpectedly.
   Please try again.
   

   

   
   
9. Extra: If the size of the emulator is too big. You can resize it by:
   Run > Run Configurations > Target [a tab]
   Write in Additional Emulator Command Line Options: "-scale 0.8" or another number

I tried debugging that "stopped unexpectedly" error for a long time. Here are the things I learned:

Finding the base java file: I tried to match up with  the tutorial by grepping for "extends Activity". I found it in the org/thialfihar.android.apg/BaseActivity.java file, which also has onCreate().

Debugger can be useless: I tried placing a break in the first line of the main function in the basefile. When I ran Run > Debug History and started stepping through the files, a lot of Source not found came up. It was misleading, because when I tried breaking in similar fashions in a working "Hello Android" project, the same "Source not found" errors came up.

LogCat is supposed to help debugging, but since I had no idea what the code was about, it wasn't helpful for me.