Getting ethernet_ping to work

Goal: recreate scratchpad's ether_ping in my ethernet gem.

AF / PF POSIX Socket API

Ruby learning
ruby array pack() array-->binary string
string unpack string --> array
array first()
attr_reader :to_return is an alias for returning that instance's to_return.

ether_ping  eth0                88B7                       001122334455        aa00112233
ether_ping  net_interface   ether_type 2bytes   dest_mac 6byte        data 0pad to 64 bytes

scratchpad's ether_ping, relies on
- scratchpad/bin/
        - ether_ping is the client
                  client = Scratchpad::Ethernet::PingClient.new interface, ether_type, dest_mac
                  puts client.ping(data)
        - ether_ping_server
                  it seems that calling ether_ping doesn't need ether_ping_server's stuff
- scratchpad/lib/scratchpad/ethernet
        - ping.rb has classes: PingServer and PingClient
                 has require 'eventmachine' for communication
        - raw_ethernet.rb connects to the raw_ethernet stuff

What I did:
1. keep looking at that. Especially socket http://www.ruby-doc.org/stdlib/libdoc/socket/rdoc/classes/Socket.html
2. ethernet is basically ether_shell renamed, with scratchpad's pinging
3. Put in system-getifaddr in too

trying

ethernet$ rake
(in /home/haoqili/Desktop/rails/ethernet)
/usr/bin/ruby1.8 -S bundle exec rspec "spec/raw_socket_spec.rb" "spec/ethernet_spec.rb" "spec/socket_wrapper_spec.rb"
/usr/lib/ruby/gems/1.8/gems/rspec-core-2.3.1/lib/rspec/core/backward_compatibility.rb:20:in `const_missing': uninitialized constant ShellStub::Ethernet (NameError)
from /usr/lib/ruby/gems/1.8/gems/rspec-expectations-2.3.0/lib/rspec/expectations/backward_compatibility.rb:6:in `const_missing'
from /home/haoqili/Desktop/rails/ethernet/spec/support/shell_stub.rb:2
from /home/haoqili/Desktop/rails/ethernet/spec/spec_helper.rb:8:in `require'
from /home/haoqili/Desktop/rails/ethernet/spec/spec_helper.rb:8
from /home/haoqili/Desktop/rails/ethernet/spec/spec_helper.rb:8:in `each'
from /home/haoqili/Desktop/rails/ethernet/spec/spec_helper.rb:8
from /home/haoqili/Desktop/rails/ethernet/spec/raw_socket_spec.rb:4:in `require'
from /home/haoqili/Desktop/rails/ethernet/spec/raw_socket_spec.rb:4
from /usr/lib/ruby/gems/1.8/gems/rspec-core-2.3.1/lib/rspec/core/configuration.rb:388:in `load'
from /usr/lib/ruby/gems/1.8/gems/rspec-core-2.3.1/lib/rspec/core/configuration.rb:388:in `load_spec_files'
from /usr/lib/ruby/gems/1.8/gems/rspec-core-2.3.1/lib/rspec/core/configuration.rb:388:in `map'
from /usr/lib/ruby/gems/1.8/gems/rspec-core-2.3.1/lib/rspec/core/configuration.rb:388:in `load_spec_files'
from /usr/lib/ruby/gems/1.8/gems/rspec-core-2.3.1/lib/rspec/core/command_line.rb:18:in `run'
from /usr/lib/ruby/gems/1.8/gems/rspec-core-2.3.1/lib/rspec/core/runner.rb:55:in `run_in_process'
from /usr/lib/ruby/gems/1.8/gems/rspec-core-2.3.1/lib/rspec/core/runner.rb:46:in `run'
from /usr/lib/ruby/gems/1.8/gems/rspec-core-2.3.1/lib/rspec/core/runner.rb:10:in `autorun'
from /usr/bin/rspec:19
rake aborted!
ruby -S bundle exec rspec "spec/raw_socket_spec.rb" "spec/ethernet_spec.rb" "spec/socket_wrapper_spec.rb" failed

(See full trace by running task with --trace)

ethernet$ sudo rake install
(in /home/haoqili/Desktop/rails/ethernet)
Successfully built RubyGem
Name: ethernet
Version: 0.0.0
File: ethernet-0.0.0.gem
Executing "ruby1.8 -S gem install ./pkg/ethernet-0.0.0.gem":
ruby1.8 -S gem install ./pkg/ethernet-0.0.0.gem
Successfully installed ethernet-0.0.0
1 gem installed
Installing ri documentation for ethernet-0.0.0...
Installing RDoc documentation for ethernet-0.0.0...


========
added "s.executables =["ethernet_ping"]" in ethernet.gemspec

/ethernet$ sudo rake install
[sudo] password for haoqili:
(in /home/haoqili/Desktop/rails/ethernet)
Successfully built RubyGem
Name: ethernet
Version: 0.0.0
File: ethernet-0.0.0.gem
Executing "ruby1.8 -S gem install ./pkg/ethernet-0.0.0.gem":
ruby1.8 -S gem install ./pkg/ethernet-0.0.0.gem
Building native extensions. This could take a while...
Successfully installed eventmachine-0.12.10
Successfully installed ethernet-0.0.0
2 gems installed
Installing ri documentation for eventmachine-0.12.10...
Installing ri documentation for ethernet-0.0.0...
Installing RDoc documentation for eventmachine-0.12.10...
Could not find main page README
Could not find main page README
Could not find main page README
Could not find main page README
Installing RDoc documentation for ethernet-0.0.0...


still not found
ethernet$ ethernet_ping
ethernet_ping: command not found

deleted s.executables = ["ethernet_ping"] from ethernet.gemspec
added gem.executables = ["ethernet_ping"] to Rakefile
$ sudo rake install
[sudo] password for haoqili:
(in /home/haoqili/Desktop/rails/ethernet)
Successfully built RubyGem
Name: ethernet
Version: 0.0.0
File: ethernet-0.0.0.gem
Executing "ruby1.8 -S gem install ./pkg/ethernet-0.0.0.gem":
ruby1.8 -S gem install ./pkg/ethernet-0.0.0.gem
Building native extensions. This could take a while...
Successfully installed eventmachine-0.12.10
Successfully installed ethernet-0.0.0
2 gems installed
Installing ri documentation for eventmachine-0.12.10...
Installing ri documentation for ethernet-0.0.0...
Installing RDoc documentation for eventmachine-0.12.10...
Could not find main page README
Could not find main page README
Could not find main page README
Could not find main page README
Installing RDoc documentation for ethernet-0.0.0...

haoqili@stay-focused-hq:18:32:11:~/Desktop/rails/ethernet$ ethernet_ping
Usage: /usr/bin/ethernet_ping net_interface ether_type dest_mac data
net_interface: name of the Ethernet interface, e.g. eth0
ether_type: packet type for the Ethernet II frame, in hex (2 bytes)
dest_mac: destination MAC for the ping packets, in hex (6 bytes)
data: ping packet data, in hex (0-padded to 64 bytes)

haoqili@stay-focused-hq:18:35:52:~/Desktop/rails/ethernet$ sudo ethernet_ping eth0 88B7 001122334455 aa00112233
/usr/lib/ruby/gems/1.8/gems/ethernet-0.0.0/bin/ethernet_ping:22: uninitialized constant Ethernet (NameError)
from /usr/bin/ethernet_ping:19:in `load'
from /usr/bin/ethernet_ping:19

19 load Gem.bin_path('ethernet', 'ethernet_ping', version)

That doesn't look right

* After Victor told me to add ethernet.rb (and then I did sudo rake install), the Ethernet NameError is gone, but now I have

$ sudo ethernet_ping eth0 88B7 001122334455 aa00112233
[sudo] password for haoqili:
/usr/lib/ruby/gems/1.8/gems/ethernet-0.0.0/bin/ethernet_ping:22: uninitialized constant Ethernet::PingClient (NameError)
from /usr/bin/ethernet_ping:19:in `load'
from /usr/bin/ethernet_ping:19

* Ah, because I never put in the file about ethernet pinging in "ethernet.rb", so I added this line:
require 'ping.rb' # This is to get ethernet_ping working :P

sudo rake install again
$ sudo ethernet_ping eth0 88B7 001122334455 aa00112233
/usr/lib/ruby/gems/1.8/gems/ethernet-0.0.0/lib/ping.rb:63:in `initialize': undefined method `socket' for Ethernet:Module (NoMethodError)
from /usr/lib/ruby/gems/1.8/gems/ethernet-0.0.0/bin/ethernet_ping:22:in `new'
from /usr/lib/ruby/gems/1.8/gems/ethernet-0.0.0/bin/ethernet_ping:22
from /usr/bin/ethernet_ping:19:in `load'
from /usr/bin/ethernet_ping:19

This is because I put my "socket" method inside the RawSocket module as well, so for methods ".socket" and ".mac", I have to change their prefix from "Ethernet::" to "Ethernet::RawSocket".


I also reordered where the ruby files are stored. Victor told me that the way he set ether_shell up is how it's done by convention. "IDEs have shortcuts that let you navigate between code and spec, as long as things stay where they're supposed to be."

so after "sudo rake install", I tried the above ethernet_ping, but I got another error:

$ sudo ethernet_ping eth0 88B7 001122334455 aa00112233
/usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb:36:in `gem_original_require': no such file to load -- ethernet/raw_socket.rb (LoadError)
from /usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb:36:in `require'
from /usr/lib/ruby/gems/1.8/gems/ethernet-0.0.0/lib/ethernet.rb:6
from /usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb:36:in `gem_original_require'
from /usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb:36:in `require'
from /usr/lib/ruby/gems/1.8/gems/ethernet-0.0.0/bin/ethernet_ping:3
from /usr/bin/ethernet_ping:19:in `load'
from /usr/bin/ethernet_ping:19

This is at commit 1b6daf320e82219a15d52d734dcbb10aff1b5fb6.

The next day, I went to see Victor. He told me to check:

$ ls /usr/lib/ruby/gems/1.8/gems/ethernet-0.0.0/lib
ethernet.rb

Indeed, the ethernet dir was not there.

I did again:

ethernet$ sudo rake install
(in /home/haoqili/Desktop/rails/ethernet)
  Successfully built RubyGem
  Name: ethernet
  Version: 0.0.0
  File: ethernet-0.0.0.gem
Executing "ruby1.8 -S gem install ./pkg/ethernet-0.0.0.gem":
ruby1.8 -S gem install ./pkg/ethernet-0.0.0.gem
Building native extensions.  This could take a while...
Successfully installed eventmachine-0.12.10
Successfully installed ethernet-0.0.0
2 gems installed
Installing ri documentation for eventmachine-0.12.10...
Installing ri documentation for ethernet-0.0.0...
Installing RDoc documentation for eventmachine-0.12.10...
Could not find main page README
Could not find main page README
Could not find main page README
Could not find main page README

$ ls /usr/lib/ruby/gems/1.8/gems/ethernet-0.0.0/lib
ethernet  ethernet.rb
haoqili@stay-focused-hq:15:16:56:~/Desktop/rails/ethernet$ sudo ethernet_ping eth0 88B7 001122334455 aa00112233
Pinging 001122334455... 

Fixed!

Shoot. It didn't show up on Wireshark :/

I printed out the source mac, sure it doesn't match eth0's mac address:
eth0 Link encap:Ethernet HWaddr 00:21:70:92:5b:c5

57 class PingClient
 58   def initialize(if_name, ether_type, destination_mac)
 59     @socket = Ethernet.socket if_name, ether_type
 60 
 61     @source_mac = [Ethernet.get_interface_mac(if_name)].pack('H*')[0, 6]
 62     puts "\n the source_mac, is"
 63     puts @source_mac.unpack('H*')[0]

ethernet's result:

$ sudo ethernet_ping eth0 88B7 001122334455 aa00112233

 the source mac is:
0192b5
Pinging 001122334455... 

scratchpad's result: (scratchpad has the working ether_ping)

$ sudo ether_ping eth0 88B7 001122334455 aa00112233

 the source_mac, is
002170925bc5
Pinging 001122334455... 

This is because ether_shell's raw_socket's .mac() doesn't have "unpack('H*').first", so if I put this back in, pinging would work in ethernet

raw_socket.rb
 32   def self.mac(eth_device)
 33     case RUBY_PLATFORM
 34     when /linux/
 35       # /usr/include/net/if.h, structure ifreq
 36       ifreq = [eth_device].pack 'a32'
 37       # 0x8927 is SIOCGIFHWADDR in /usr/include/bits/ioctls.h
 38       socket.ioctl 0x8927, ifreq
 40       #ifreq[18, 6] #in scratchpad's raw_ethernet, then says .unpack('H*').first
 41                    # because raw socket should return raw data
 42                    # leaving higher layers for presentation issues
 43     else

add it in here instead

ping.rb
66     @source_mac = [Ethernet::RawSocket.mac(eth_device).unpack('H*').first].pack('H*')[0, 6]

Yay worked!

commit: bff1c0ca79ed679aae77e5e6199c2aaf3e75fdc9

:) ethernet_ping works! Try with 'sudo ethernet_ping eth0 88B7 001122334455 aa00112233' and see it in Wireshark's eth0

How Wireshark lists devices

Goal: find how Wireshark gets the list of MAC addresses at the beginning (Windows, MacOS)
Edit: Later found out that neither Wireshark nor libpcap gives a list of MAC addresses.
Answer: To get a list of internet device names, Wireless uses dumpcap's pcap_findalldevs().

I. I looked for MAC address
II. I looked for just the simple list of devices Wireshark shows

I. I grep-ed for "MAC address" and found over 500 entries, under these big categories：
- epan/dissectors (huge)
- plugins
- airpcap

Interesting things found:

• Ulf Lamping, ulf.lamping[at]web.de, put the source and destination MAC addresses into the top-level item for Ethernet.
• "manuf" contains Ethernet vendor codes (i.e. the first half of MAC Addresses), and well-known MAC addresses. So I checked the MAC Addresses from my ifconfig -a. eth0's first half is 00:21:70, which manuf says is Dell (yes I have a Dell computer). vmnet*'s first half is 00:50:56, which manuf says is VMware. wlan0's is 00:1c:26, which manuf says is Hon Hai Precision Ind. Co. Cool.
  
• Looking at epan/dissectors doesn't seem right. Hmm, 
• Talked to Victor about it, he told me to look at the RubyGem (see other post) first

II. Give up. So what if I just track it down? The list at the beginning of Wireshark appears below the words "Start capture on interface:". So I grep-ed for those words and found that it appears in ./gtk/main_welcome.c.

There are 2 ways to go in main_welcome.c
1. WIRESHARK_STOCK_CAPTURE_INTERFACES from clicking on "Interface List" button on the main page. This is from

812 welcome_button(WIRESHARK_STOCK_CAPTURE_INTERFACES,
813         "Interface List",
814         "Live list of the capture interfaces\n(counts incoming packets)", ...

But searching down this path wasn't very fruitful

2a. By looking at how the list of devices is generated directly with the welcome_if_panel_load() function.

653 /* load the list of interfaces */
654 static void
655 welcome_if_panel_load(void)

661     GList         *if_list;
662     int err;
663     gchar         *err_str = NULL;


669     /* LOAD THE INTERFACES */

670     if_list = capture_interface_list(&err, &err_str);

btw, the g_blah() functions come from GLib

2b. So I now grep-ed for capture_interface_list(), which attempts to open all adapters it finds in order to check whether they can be captured on.

capture_interface_list() in ./capture_ifinfo.c

 1 /* capture_ifinfo.c
  2  * Routines for getting interface information from dumpcap
 23  */
 24
 25 #ifdef HAVE_CONFIG_H
 26 # include "config.h"
 27 #endif
 28
 29 #ifdef HAVE_LIBPCAP
 30
 31 #include <stdlib.h>
 32 #include <string.h>
 33
 34 #ifdef HAVE_ARPA_INET_H
 35 #include <arpa/inet.h>
 36 #endif
 37
 38 #ifdef HAVE_SYS_SOCKET_H
 39 #include <sys/socket.h>         /* needed to define AF_ values on UNIX */
 40 #endif
 41
 42 #ifdef HAVE_WINSOCK2_H
 43 #include <winsock2.h>           /* needed to define AF_ values on Windows */

 44 #endif

 45 

 46 #ifdef NEED_INET_V6DEFS_H

 47 # include "wsutil/inet_v6defs.h"

 48 #endif

 49 

 50 #include <glib.h>

 51 

 52 #include "capture_opts.h"

 53 #include "capture_sync.h"

 54 #include "log.h"

 55 

 56 #include "capture_ifinfo.h"

 57 

 58 /**

 59  * Fetch the interface list from a child process (dumpcap).

 60  *

 61  * @return A GList containing if_info_t structs if successful, NULL (with err and possibly err_str set) otherwise.

 62  *

 63  */

 64 

 65 /* XXX - We parse simple text output to get our interface list.  Should

 66  * we use "real" data serialization instead, e.g. via XML? */

 67 GList *

 68 capture_interface_list(int *err, char **err_str)

 69 {

 70     int        ret;

 71     GList     *if_list = NULL;

 72     int        i, j;

 73     gchar     *data, *primary_msg, *secondary_msg;

 74     gchar    **raw_list, **if_parts, **addr_parts;

 75     gchar     *name;

 76     if_info_t *if_info;

 77     if_addr_t *if_addr;

 78 

 79     g_log(LOG_DOMAIN_CAPTURE, G_LOG_LEVEL_MESSAGE, "Capture Interface List ...");

 80 

 81     /* Try to get our interface list */

 82     ret = sync_interface_list_open(&data, &primary_msg, &secondary_msg);

 83     if (ret != 0) {

 84         g_log(LOG_DOMAIN_CAPTURE, G_LOG_LEVEL_MESSAGE, "Capture Interface List failed!");

 85         if (err_str) {

 86             *err_str = primary_msg;

 87         } else {

 88             g_free(primary_msg);

 89         }

 90         g_free(secondary_msg);

 91         *err = CANT_GET_INTERFACE_LIST;

 92         return NULL;

 93     }

 94 

 95     /* Split our lines */

 96 #ifdef _WIN32

 97     raw_list = g_strsplit(data, "\r\n", 0);

 98 #else

 99     raw_list = g_strsplit(data, "\n", 0);

100 #endif

101     g_free(data);

102 

103     for (i = 0; raw_list[i] != NULL; i++) {

104         if_parts = g_strsplit(raw_list[i], "\t", 4);

105         if (if_parts[0] == NULL || if_parts[1] == NULL || if_parts[2] == NULL ||

106                 if_parts[3] == NULL) {

107             g_strfreev(if_parts);

108             continue;

109         }

110 

111         /* Number followed by the name, e.g "1. eth0" */

112         name = strchr(if_parts[0], ' '); /*strchr searches to see if there is a single
                                                                space chr in if_parts[0] */

113         if (name) {

114             name++;

115         } else {

116             g_strfreev(if_parts);

117             continue;

118         }

119 

120         if_info = g_malloc0(sizeof(if_info_t));

121         if_info->name = g_strdup(name);

122         if (strlen(if_parts[1]) > 0)

123             if_info->description = g_strdup(if_parts[1]);

124         addr_parts = g_strsplit(if_parts[2], ",", 0);

125         for (j = 0; addr_parts[j] != NULL; j++) {

126             if_addr = g_malloc0(sizeof(if_addr_t));

127             if (inet_pton(AF_INET, addr_parts[j], &if_addr->addr.ip4_addr)) {

128                 if_addr->ifat_type = IF_AT_IPv4;

129             } else if (inet_pton(AF_INET6, addr_parts[j],

130                     &if_addr->addr.ip6_addr)) {

131                 if_addr->ifat_type = IF_AT_IPv6;

132             } else {

133                 g_free(if_addr);

134                 if_addr = NULL;

135             }

136             if (if_addr) {

137                 if_info->addrs = g_slist_append(if_info->addrs, if_addr);

138             }

139         }

140         if (strcmp(if_parts[3], "loopback") == 0)

141             if_info->loopback = TRUE;

142         g_strfreev(if_parts);

143         g_strfreev(addr_parts);

144         if_list = g_list_append(if_list, if_info);

145     }

146     g_strfreev(raw_list);

147 

148     /* Check to see if we built a list */

149     if (if_list == NULL) {

150         *err = NO_INTERFACES_FOUND;

151         if (err_str)

152             *err_str = g_strdup("No interfaces found");

153     }

154     return if_list;

155 }

2c. Then look at: sync_interface_list_open(), which is found in ./capture_sync.c:

1073 /*
1074  * Get the list of interfaces using dumpcap.
1075  *
1076  * On success, *data points to a buffer containing the dumpcap output,
1077  * *primary_msg and *secondary_msg are NULL, and 0 is returned.  *data
1078  * must be freed with g_free().
1079  *
1080  * On failure, *data is NULL, *primary_msg points to an error message,
1081  * *secondary_msg either points to an additional error message or is
1082  * NULL, and -1 is returned; *primary_msg, and *secondary_msg if not NULL,
1083  * must be freed with g_free().
1084  */
1085 int
1086 sync_interface_list_open(gchar **data, gchar **primary_msg,
1087                          gchar **secondary_msg)
1088 {
1089     int argc;
1090     const char **argv;
1091
1092     g_log(LOG_DOMAIN_CAPTURE, G_LOG_LEVEL_DEBUG, "sync_interface_list_open");
1093
1094     argv = init_pipe_args(&argc);
1095
1096     if (!argv) {
1097         *primary_msg = g_strdup("We don't know where to find dumpcap.");
1098         *secondary_msg = NULL;
1099         *data = NULL;
1100         return -1;
1101     }
1102
1103     /* Ask for the interface list */
1104     argv = sync_pipe_add_arg(argv, &argc, "-D");
1105
1106 #ifndef DEBUG_CHILD
1107     /* Run dumpcap in capture child mode */
1108     argv = sync_pipe_add_arg(argv, &argc, "-Z");
1109     argv = sync_pipe_add_arg(argv, &argc, SIGNAL_PIPE_CTRL_ID_NONE);
1110 #endif
1111     return sync_pipe_run_command(argv, data, primary_msg, secondary_msg);
1112 }

It's passing in an argument to get the devices, and the argument is "dumpcap"

2d. dumpcap and dumpcap.c
dumpcap is cross platform. :)
man dumpcap told me that the option "-D" prints a list of the interfaces on which Dumpcap can capture.

I found dumpcap's code in "dumpcap.c":

3792      * "-D" requires no interface to be selected; it's supposed to list
3793      * all interfaces.
3794      */
3795     if (list_interfaces) {
3796         /* Get the list of interfaces */
3797         GList       *if_list;
3798         int         err;
3799         gchar       *err_str;
3800 
3801         if_list = capture_interface_list(&err, &err_str);

So then I track down this function

796 GList *
 797 capture_interface_list(int *err, char **err_str)
 798 {
 799     return get_interface_list(err, err_str);
 800 }

after "grep -ir get_interface_list ." I found it in:

2e. capture-wpcap.c

676 /*
677  * This will use "pcap_findalldevs()" if we have it, otherwise it'll
678  * fall back on "pcap_lookupdev()".
679  */
680 GList *
681 get_interface_list(int *err, char **err_str)
682 {
683     GList  *il = NULL;
684     wchar_t *names;
685     char *win95names;
686     char ascii_name[MAX_WIN_IF_NAME_LEN + 1];
687     char ascii_desc[MAX_WIN_IF_NAME_LEN + 1];
688     int i, j;
689     char errbuf[PCAP_ERRBUF_SIZE];
690 
691 #ifdef HAVE_PCAP_FINDALLDEVS
692     if (p_pcap_findalldevs != NULL)
693         return get_interface_list_findalldevs(err, err_str);
694 #endif
...
738 
739     names = (wchar_t *)pcap_lookupdev(errbuf);

There are 2 methods, get_interface_list_findalldevs() or pcap_lookupdev()

Both methods will require code from libpcap, which I found on this site.

Working out the 2 different ways

2d.1. First, let's try with get_interface_list_findalldevs()
Inside capture-pcap-util.c

107 #ifdef HAVE_PCAP_FINDALLDEVS
...
166 GList *
167 get_interface_list_findalldevs(int *err, char **err_str)
168 {
169     GList  *il = NULL;
170     pcap_if_t *alldevs, *dev;
171     if_info_t *if_info;
172     char errbuf[PCAP_ERRBUF_SIZE];
173 
174     if (pcap_findalldevs(&alldevs, errbuf) == -1) {
175         *err = CANT_GET_INTERFACE_LIST;
176         if (err_str != NULL)
177             *err_str = cant_get_if_list_error_message(errbuf);
178         return NULL;
179     }
180 
181     if (alldevs == NULL) {
182         /*
183          * No interfaces found.
184          */
185         *err = NO_INTERFACES_FOUND;
186         if (err_str != NULL)
187             *err_str = NULL;
188         return NULL;
189     }
190 
191     for (dev = alldevs; dev != NULL; dev = dev->next) {
192         if_info = if_info_new(dev->name, dev->description);
193         il = g_list_append(il, if_info);
194         if_info_ip(if_info, dev);
195     }
196     pcap_freealldevs(alldevs);
197 
198     return il;
199 }
200 #endif /* HAVE_PCAP_FINDALLDEVS */

2d.2 Now, let's try with the other method pcap_lookupdev()

233 char*
234 pcap_lookupdev (char *a)
235 {
236     if (!has_wpcap) {
237         return NULL;
238     }
239     return p_pcap_lookupdev(a);
240 }

which is a pointer

56 static char*   (*p_pcap_lookupdev) (char *);

2d.2b. Now we have to look at libpcap's code for pcap_lookupdev, which will also lead to

pcap_findalldevs()

Found it in inet.c

635 #if !defined(WIN32) && !defined(MSDOS)
636 
637 /*
638  * Return the name of a network interface attached to the system, or NULL
639  * if none can be found.  The interface must be configured up; the
640  * lowest unit number is preferred; loopback is ignored.
641  */
642 char *
643 pcap_lookupdev(errbuf)
644     register char *errbuf;
645 {
...
654     if (pcap_findalldevs(&alldevs, errbuf) == -1)

777 #elif defined(WIN32)
778 
779 /*
780  * Return the name of a network interface attached to the system, or NULL
781  * if none can be found.  The interface must be configured up; the
782  * lowest unit number is preferred; loopback is ignored.
783  */
784 char *
785 pcap_lookupdev(errbuf)
786     register char *errbuf;
787 {
788     DWORD dwVersion;
789     DWORD dwWindowsMajorVersion;
790     dwVersion = GetVersion();   /* get the OS version */
791     dwWindowsMajorVersion = (DWORD)(LOBYTE(LOWORD(dwVersion)));
792 
793     if (dwVersion >= 0x80000000 && dwWindowsMajorVersion >= 4) {
794         /*
795          * Windows 95, 98, ME.
796          */
797         ULONG NameLength = 8192;
798         static char AdaptersName[8192];
799 
800         if (PacketGetAdapterNames(AdaptersName,&NameLength) )
801             return (AdaptersName);
802         else
803             return NULL;
804     } else {
805         /*
806          * Windows NT (NT 4.0, W2K, WXP). Convert the names to UNICODE for backward compatibiliy

3. Now we need to look inside libpcap-1.1.1 for pcap_findalldevs()

Method description from pcap-sita.html:
SMP The Supervisory Management Processor where Wireshark (or equivalent) runs in conjuction with a libpcap front-end.
IOP I/O Processors where the monitored ports exist in conjunction with a custom device driver/libpcap back-end.

pcap_findalldevs constructs a list of network devices that can be opened with pcap_open_live().

SMP It obtains a list of IOPs currently available (via /etc/hosts).
SMP -> IOP The SMP will sequentially open a connection to each IOP on its 'sniffer' port to ensure the IOP is available. It sends a null terminated empty interface ID followed by the query request command.
IOP -> SMP The IOP returns an error response and its list of devices.
SMP -> IOP The SMP closes the TCP connection with each IOP.
SMP The SMP adds the received information to its internal structure.

=========================
SMP/IOP Inter-Process Communication Protocol

Communications between an SMP and an IOP consists of a TCP session
between an ephemeral port on the SMP and the well known port of 49152
(which is the first available port in the 'dynamic and/or private port'
range) on an IOP.

Following a TCP open operation the IOP receives a null terminated
'interface ID' string to determine the type of operation that follows:

Every command received by an IOP implies a 'stop trace/stop forwarding' operation must
occur before executing the received command.

A session is closed when the SMP closes the TCP session with the IOP.
Obviously monitoring and forwarding is also stopped at that time.

Note: All multi-octet entities are sent in network neutral order.

SMP -> IOP	Open socket (to each IOP), and sends: Name/ Purpose Size (in bytes) Description Interface ID 1 A NULL to indicate an an empty 'interface ID'.
IOP -> SMP	Send its (possibly empty) NULL terminated error response string.
SMP -> IOP	Sends the 'interface query request': Name/ Purpose Size (in bytes) Description Interface ID 1 A 'Q' (indicating 'interface query request').
IOP -> SMP	The IOP returns a list of sequences of information as defined by the return parameter of this function call (as shown in the following table). Elements are specified by providing an unsigned byte preceeding the actual data that contains length information. Notes: Name/ Purpose Size (in bytes) Description length 1 The number of octets in the name field that follows. Name 1-255 The name of the interface. The format of the name is an alphabetic string (indicating the type of interface) followed by an optional numeric string (indicating the interface's sequence number). Sequence numbers (if needed) will begin at zero and progress monotonically upwards. (i.e. 'eth0', 'lo', 'wan0', etc.) For an IOP, the alphabetic string will be one of: 'eth', 'wan', and 'lo' for Ethernet, WAN ports and the IP loopback device respectively. An IOP currently supports: 'eth0', 'eth1', 'lo', 'wan0' ... 'wan7'. Note: IOPs and ACNs will not currently support the concept of 'any' interface. length 1 The number of octets in the interface description field that follows. Interface Description 0-255 A description of the interface or it may be an empty string. (i.e. 'ALC') Interface Type 4 The type of interface as defined in the description for pcap_datalink() (in network neutral order). Loopback Flag 1 1 = if the interface is a loopback interface, zero = otherwise. count 1 # of address entries that follow. Each entry is a series of bytes in network neutral order. See the parameter definition above for more details. Repeated 'count' number of times. length 1 The number of octets in the address field that follows. Address 1-255 The address of this interface (in network neutral order). length 1 The number of octets in the netmask field that follows. Network Mask 0-255 The network mask used on this interface (if applicable) (in network neutral order). length 1 The number of octets in the broadcast address field that follows. Broadcast Address 0-255 The broadcast address of this interface (if applicable) (in network neutral order). length 1 The number of octets in the destination address field that follows. Destination Address 0-255 The destination address of this interface (if applicable) (in network neutral order).
SMP -> IOP	Close the socket.
IOP -> SMP	Close the socket.

=========================


I have "pcap_findalldevs" source from the following files in libpcap-1.1.1:
they all take the same arguments
fad-win32.c - for Windows OS

216  * Win32 implementation, based on WinPcap
217  */
218 int
219 pcap_findalldevs(pcap_if_t **alldevsp, char *errbuf)

pcap-dos.c - for DOS (predecessor of Windows)
fad-glif.c - for Solaris
fad-gifc.c - This is the implementation used on platforms that have SIOCGIFCONF but don't have any other mechanism for getting a list of interfaces.
fad-getad.c - This is the implementation used on platforms that have "getifaddrs()"

(btw MacOS uses free bsd.)

Already have a gem to get Linux list MAC Address working
MacOS can be on hold
Focus on getting Windows to work! look at fad-win32.c in more detail

Reading Wireshark

Goal: find how Wireshark gets the MAC address on Windows and MacOS

To reach the goal:
1. finish reading Wireshark docs, see notes below
2. read victor email, see this post
3. read grep of "mac address" in doc. It was useless
4. read grep of "mac address" from wireshark root
5. email wireshark dev
6. How to get Linux MAC address? look in ether_shell

At first, I grep-ed for "MAC address" in the entire Wireshark directory with grep -ir "mac address" . > hq_macaddsearch.txt. It took way to long to complete so I stopped it. Then I looked that it already had over 150,000 lines where the bottom is filled with "hq_macaddsearch.txt" ... oh I made the infinite loop mistake again, should have saved it to ../hq_macaddsearch.txt :/

I grep-ed for "Mac address" in the entire Wireshark directory with grep -ir "mac address" . and found over 500 entries. I would go back to it after reading the docs to understand which entries are important.

Reading Wireshark docs:

• The overview of the Wireshark
  
  • Dumpcap needs elevated privileges to do the capturing. It has code customized for each OS. It talks with WinPcap or libpcap, platform-dependent packet capture libraries to capture from different types of interfaces.
    
• In the different dissectors in epan/dissectors decodes a layer of the protocol before handing it to the next level of dissector.
• Wireshark is based on the GTK toolkit for the UI.
• Not very helpful in trying to find MAC addresses

epan = Ethereal Packet ANalyzer
ett = Ethereal Tree Type
hf = header field
gchar* = pointer to character string
guint8 = 1-byte value
guint16 = 2-byte value
tvb = Testy Virtual Buffer

Reading Wireshark & Ethereal Network Protocol Analyzer Toolkit

• every NIC (Network Interface Card) has a unique MAC Address
• MAC adress is a 48-bit number, aka 12-digit hexadecimal number. The first half identifies the manufacturer of the Ethernet card, last 24 bits is a serial number assigned. e.g.:12:34:56:78:9a:bc
• View MAC Address on different operating systems:
• Windows XP: command line type ipconfig /all, under Physical Address
  
• Linux: command line type ifconfig -a, listed as "HWaddr". "lo" doesn't have a MAC address, because it's a loop back address only. On other computersarp -a might be useful too
  
• MacOS: command line type ifconfig -a, listed as "Ether" label.

Wireshark Source Code to help out

SVN Get the source code with SVN. Code can also be viewed online.

Resources:
- Wireshark Devolpement Guide, especially read this section on Wireshark Developement
- Wireshark Development Wiki
- doc/README.developer
- Wireshark User's Guide

Wireshark uses libpcap (like WinPcap) to do packet capture.

Old to-do for Wireshark (ranked in terms of easiness)
a) find where it communicates to ethernet
--1 read the section on Wireshark Developement
b) MAC addresses
--2 read how the Victor's EtherShell gets Mac address for Linux
--3 get MAC addresses in Windows and MacOS
(4) ... if I run into trouble, set up Wireshark on a vm to test it

Testing out the old ethernet ping through Wireshark

Currently I have an empty ethernet gem. So after talking to Victor (thank you! quotes in here are Victor's words), I tested the old ethernet code before sticking it into my gem..

The big picture: Create some bytes from my mac address to flow through the ethernet card, and wireshark should capture this flow.

Today I tested out what Victor already wrote, for a minimum ethernet pinging. Later I should take his code and improve on it.

Outline of Testing Victor's Ethernet Pinging Code

1. Install the scratchpad rubygem with rake install.
2. Try the ether_ping command
3. See if the ping is caught on Wireshark

More details on 2:
bin/ether_ping calls the ethernet code. When you install the gem, you get the files in its bin/ on your path. So the bin/ file
- sets things up
- reads command-line args
- and calls up the ethernet code

More details on 3:
Wireshark is used to debug, and you can click on the packets to see the data. All you need is the basic functionality of capturing.
I asked a question: Shouldn't the data be coming/going from the server or the P chip?
Victor answened: Layers. The ethernet layer doesn't care about who sends data to whom. The ethernet layer takes an ethernet card, a mac address, a bunch of bytes and puts the bytes on the wire.

Details of Testing Victor's Ethernet Pinging Code (highlighted commands are the final steps, skipping over me talking about errors)

1. git pull in my local copy of Victor's scratchpad rubygem to update it. (Your latest entry from git log should match the newest update on scratchpad's github version)
   
2. rake install
   
   
   (in /home/haoqili/Desktop/rails/secureSt/scratchpad)
   Successfully built RubyGem
   Name: scratchpad
   Version: 0.0.1
   File: scratchpad-0.0.1.gem
   Executing "ruby -S gem install ./pkg/scratchpad-0.0.1.gem":
   ruby -S gem install ./pkg/scratchpad-0.0.1.gem
   Fetching: eventmachine-0.12.10.gem (100%)
   Building native extensions. This could take a while...
   Fetching: ffi-1.0.7.gem (100%)
   Building native extensions. This could take a while...
   Fetching: json-1.5.1.gem (100%)
   Building native extensions. This could take a while...
   Successfully installed eventmachine-0.12.10
   Successfully installed ffi-1.0.7
   Successfully installed json-1.5.1
   Successfully installed scratchpad-0.0.1
   4 gems installed
   Installing ri documentation for eventmachine-0.12.10...
   Installing ri documentation for ffi-1.0.7...
   Installing ri documentation for json-1.5.1...
   Installing ri documentation for scratchpad-0.0.1...
   Installing RDoc documentation for eventmachine-0.12.10...
   Installing RDoc documentation for ffi-1.0.7...
   Installing RDoc documentation for json-1.5.1...
   Installing RDoc documentation for scratchpad-0.0.1...
   
   I was puzzled why it says 4 gems installed even though scratchpad only has 1 .gemspec. Victor told me that "it probably needed to install dependencies since this is the first time you installed it".
   
3. Type ether_ping, and the output tells you how to use it:
   
   Usage: /home/haoqili/.rvm/gems/ruby-1.9.2-p136/bin/ether_ping net_interface ether_type dest_mac data
   net_interface: name of the Ethernet interface, e.g. eth0
   ether_type: packet type for the Ethernet II frame, in hex (2 bytes)
   dest_mac: destination MAC for the ping packets, in hex (6 bytes)
   data: ping packet data, in hex (0-padded to 64 bytes)
4. Victor gave me ether_ping eth0 88B7 001122334455 aa00112233, which was his test command for ether_ping. The ether_type Ethernet II frame is "88B7", which is "in the range of experimental protocols, but you can pretty much use anything that's not IP and you won't confuse your system." He also told me all the other exeprimental etherent II frames "are 4 numbers somewhere around the one I picked.". So I tried ether_ping eth0 88B7 001122334455 aa00112233, which gave:
   
   /home/haoqili/.rvm/gems/ruby-1.9.2-p136/gems/scratchpad-0.0.1/lib/scratchpad/ethernet/raw_ethernet.rb:11:in `initialize': Operation not permitted - socket(2) (Errno::EPERM)
   from /home/haoqili/.rvm/gems/ruby-1.9.2-p136/gems/scratchpad-0.0.1/lib/scratchpad/ethernet/raw_ethernet.rb:11:in `new'
   from /home/haoqili/.rvm/gems/ruby-1.9.2-p136/gems/scratchpad-0.0.1/lib/scratchpad/ethernet/raw_ethernet.rb:11:in `socket'
   from /home/haoqili/.rvm/gems/ruby-1.9.2-p136/gems/scratchpad-0.0.1/lib/scratchpad/ethernet/ping.rb:59:in `initialize'
   from /home/haoqili/.rvm/gems/ruby-1.9.2-p136/gems/scratchpad-0.0.1/bin/ether_ping:21:in `new'
   from /home/haoqili/.rvm/gems/ruby-1.9.2-p136/gems/scratchpad-0.0.1/bin/ether_ping:21:in `<top (required)="">'
   from /home/haoqili/.rvm/gems/ruby-1.9.2-p136/bin/ether_ping:19:in `load'
   </top>
   
   Puzzled at the output, Victor told me to "sudo", because an "EMPERM is an OS-level error", not something that's wrong with the command I put. I then attepted sudo ether_ping eth0 88B7 001122334455 aa00112233, but it failed as well.
   
   [sudo] password for haoqili:
   sudo: ether_ping: command not found
   • bin/enable_pcacp: Victor said btw that "in the scratchpad gem there's a script [bin/enable_pcacp] that gives Ruby the right access bits so you don't have to sudo all the time to talk to the ethernet."
     
   • After I did sudo enable_pcap, I got: [sudo] password for haoqili: sudo: enable_pcap: command not found.
     
   • Victor: it's issuing setcap. So try setcap and see what it tells you to install: sudo setcap, output-ing sudo setcap usage: blahbalh Note <filename> must be a regular (non-symlink) file.
     
   What could be wrong with ether_ping that neither sudo-ing it or not works? Continue reading ...
5. I tried which ether_ping, yielding /home/haoqili/.rvm/gems/ruby-1.9.2-p136/bin/ether_ping, instead of /usr/bin/ether_ping! CULPRINT FOUND! Fixing this should fix ether_ping! But it took me a while to fix it.
6. ether_ping is pointed to the wrong location because I was using rvm (Victor is so smart), "so it installed scratchpad in one of your gemsets. You need to install it in the system rubygems, rvm use system, then rake install" To which I suggested of simply stop using rvm, but Victor said "rvm is useful, you'll need to make sure your thing works in 1.8.7 and 1.9.2. So it's good to use it in general. It doesn't do well with sudo though."
7. But I got confused on what to do exactly, so Victor told me for now to just "remove the rvm import from your bash profile and use the system ruby for now like you suggested" rvm --default use system note the dash is actually 2 dashes. It worked:
   
   find: warning: you have specified the -maxdepth option after a non-option argument -name, but options are not positional (-maxdepth affects tests specified before it as well as those specified after it). Please specify options before other arguments.
   
   Now using system ruby.
   find: warning: you have specified the -maxdepth option after a non-option argument -name, but options are not positional (-maxdepth affects tests specified before it as well as those specified after it). Please specify options before other arguments.
   
   Now using system ruby.
8. Now I have to rake install, but first I encountered a program when doing rake install:
   
   (in /home/haoqili/Desktop/rails/secureSt/scratchpad)
   Jeweler (or a dependency) not available. Install it with: gem install jeweler
   rake aborted!
   Don't know how to build task 'install'
   
   (See full trace by running task with --trace)
9. As suggested, I did: sudo gem install jeweler, outputting:
   
   Fetching: git-1.2.5.gem (100%)
   Fetching: jeweler-1.5.2.gem (100%)
   Successfully installed git-1.2.5
   Successfully installed jeweler-1.5.2
   2 gems installed
   Installing ri documentation for git-1.2.5...
   Installing ri documentation for jeweler-1.5.2...
   Installing RDoc documentation for git-1.2.5...
   Installing RDoc documentation for jeweler-1.5.2...
10. rake install for reals this time!
    
    (in /home/haoqili/Desktop/rails/secureSt/scratchpad)
    Successfully built RubyGem
    Name: scratchpad
    Version: 0.0.1
    File: scratchpad-0.0.1.gem
    Executing "ruby1.8 -S gem install ./pkg/scratchpad-0.0.1.gem":
    ruby1.8 -S gem install ./pkg/scratchpad-0.0.1.gem
    Fetching: eventmachine-0.12.10.gem (100%)
    Building native extensions. This could take a while...
    Fetching: ffi-1.0.7.gem (100%)
    Building native extensions. This could take a while...
    Fetching: json-1.5.1.gem (100%)
    Building native extensions. This could take a while...
    Successfully installed eventmachine-0.12.10
    Successfully installed ffi-1.0.7
    Successfully installed json-1.5.1
    Successfully installed scratchpad-0.0.1
    4 gems installed
    Installing ri documentation for eventmachine-0.12.10...
    Installing ri documentation for ffi-1.0.7...
    Installing ri documentation for json-1.5.1...
    Installing ri documentation for scratchpad-0.0.1...
    Installing RDoc documentation for eventmachine-0.12.10...
    Could not find main page README
    Could not find main page README
    Could not find main page README
    Could not find main page README
    Installing RDoc documentation for ffi-1.0.7...
    Installing RDoc documentation for json-1.5.1...
    Installing RDoc documentation for scratchpad-0.0.1...
11. ether_ping eth0 88B7 001122334455 aa00112233 still gives this error like before
    
    /usr/lib/ruby/gems/1.8/gems/scratchpad-0.0.1/lib/scratchpad/ethernet/raw_ethernet.rb:11:in `initialize': Operation not permitted - socket(2) (Errno::EPERM)
    from /usr/lib/ruby/gems/1.8/gems/scratchpad-0.0.1/lib/scratchpad/ethernet/raw_ethernet.rb:11:in `new'
    from /usr/lib/ruby/gems/1.8/gems/scratchpad-0.0.1/lib/scratchpad/ethernet/raw_ethernet.rb:11:in `socket'
    from /usr/lib/ruby/gems/1.8/gems/scratchpad-0.0.1/lib/scratchpad/ethernet/ping.rb:59:in `initialize'
    from /usr/lib/ruby/gems/1.8/gems/scratchpad-0.0.1/bin/ether_ping:21:in `new'
    from /usr/lib/ruby/gems/1.8/gems/scratchpad-0.0.1/bin/ether_ping:21
    from /usr/bin/ether_ping:19:in `load'
    from /usr/bin/ether_ping:19
12. So instead do sudo ether_ping eth0 88B7 001122334455 aa00112233
    
    Pinging 001122334455...
    looking great so far!
    
    I stop it with CTRL+C because "it's a ping tool so it expects a response back. There won't be a response because there isn't anyone listening at that MAC [001122334455] so it'll hang forever."
    
    Pinging 001122334455... ^C/usr/lib/ruby/gems/1.8/gems/scratchpad-0.0.1/lib/scratchpad/ethernet/ping.rb:82:in `recv': Interrupt
    from /usr/lib/ruby/gems/1.8/gems/scratchpad-0.0.1/lib/scratchpad/ethernet/ping.rb:82:in `ping'
    from /usr/lib/ruby/gems/1.8/gems/scratchpad-0.0.1/bin/ether_ping:26
    from /usr/lib/ruby/gems/1.8/gems/scratchpad-0.0.1/bin/ether_ping:23:in `loop'
    from /usr/lib/ruby/gems/1.8/gems/scratchpad-0.0.1/bin/ether_ping:23
    from /usr/bin/ether_ping:19:in `load'
    from /usr/bin/ether_ping:19
13. I open up Wireshark with sudo wireshark and start capturing "eth0". I run the above sudo ether_ping command again hoping something would show up, but it didn't. Again, I ask Victor to help because he knows everything about this project and is always patient with me asking questions. Victor says: "it may reject the packet because it's not formatted for wifi. Point ether_ping to a lan card, eth* [eth0 works]." But I did point it to eth0. "Maybe it doesn't work without a cable in it. I don't think I ever tested without a cable."
    
14. Plug in a Ethernet cable into your computer! Don't do wireless!
    
15. This command should work (finally!):
    sudo ether_ping eth0 88B7 001122334455 aa00112233
    
    In Wireshark, you should see:
    
    
    
    
    
    Click on it and click on "data" look! "aa00112233" is right there!!! Good job!!