Tutorial

Coping Photos from Camera to Linux with Gphoto2

 

Install gphoto2

sudo apt-get install gphoto2

Example from my camera:

===============================================

To make sure I can see camera

billf@mars $ gphoto2 –auto-detect

Model Port

———————————————————-

USB PTP Class Camera usb:004,098

 

billf@mars ~/Downloads $ gphoto2 –list-ports
Devices found: 2
Path                             Description
————————————————————–
ptpip:                           PTP/IP Connection
usb:003,012              Universal Serial Bus
   
Model                          Port
———————————————————-
USB PTP Class Camera           usb:003,012
billf@mars ~/Downloads $ gphoto2 –summary
Camera summary:
Manufacturer: Nikon Corporation
Model: L30
Version: COOLPIX L30 V1.0
Vendor Extension ID: 0xa (1.0)
Vendor Extension Description: microsoft.com: 1.0;

Capture Formats: JPEG
Display Formats: Undefined Type, Association/Directory, Defined Type, JPEG, DPOF, MS AVI, Apple Quicktime, MS Wave

Device Capabilities:
File Download, File Deletion, File Upload
Generic Image Capture, No Open Capture, No vendor specific capture
Nikon Wifi support

Storage Devices Summary:
store_00010001:
StorageDescription: L30
VolumeLabel: None
Storage Type: Removable RAM (memory card)
Filesystemtype: Digital Camera Layout (DCIM)
Access Capability: Read-Write
Maximum Capability: 15539896320 (14820 MB)
Free Space (Bytes): 13047693312 (12443 MB)
Free Space (Images): 2354

Device Property Summary:
Property 0xd407:(read only) (type=0x6) 1
Property 0xd406:(readwrite) (type=0xffff) ”
Property 0xd002:(readwrite) (type=0x6) Enumeration [1,2,3,4,5,6,7] value: 6
Date & Time(0x5011):(readwrite) (type=0xffff) ‘20150910T154012’
Flash Mode(0x500c):(readwrite) (type=0x4) Enumeration [1,2,3,4] value: Automatic flash (1)
Focus Mode(0x500a):(readwrite) (type=0x4) Enumeration [2,3] value: Automatic (2)
Focal Length(0x5008):(read only) (type=0x6) Enumeration [3500,4600,5300,6100,7300,8600,10500] value: 35 mm (3500)
Battery Level(0x5001):(read only) (type=0x2) Enumeration [2,5,25,50,65,80,100] value: 80% (80)
Property 0xd303:(read only) (type=0x2) 1

billf@mars ~/Downloads $ gphoto2 –list-files
There is no file in folder ‘/’.
There is no file in folder ‘/store_00010001’.
There is no file in folder ‘/store_00010001/DCIM’.
There are 200 files in folder ‘/store_00010001/DCIM/100NIKON’.
#1     DSCN0520.JPG               rd  8757 KB 5152×3864 image/jpeg
#2     DSCN0521.JPG               rd  8631 KB 5152×3864 image/jpeg
#3     DSCN0522.JPG               rd  8616 KB 5152×3864 image/jpeg
#4     DSCN0523.JPG               rd  8060 KB 5152×3864 image/jpeg
#5     DSCN0524.JPG               rd  8644 KB 5152×3864 image/jpeg
#6     DSCN0525.JPG               rd  8755 KB 5152×3864 image/jpeg
#7     DSCN0526.JPG               rd  7622 KB 5152×3864 image/jpeg
#8     DSCN0527.JPG               rd  7185 KB 5152×3864 image/jpeg
#9     DSCN0528.JPG               rd  8342 KB 5152×3864 image/jpeg
#10    DSCN0529.JPG               rd  8139 KB 5152×3864 image/jpeg
#11    DSCN0530.JPG               rd  8769 KB 5152×3864 image/jpeg
#12    DSCN0531.JPG               rd  8888 KB 5152×3864 image/jpeg
#13    DSCN0532.JPG               rd  8759 KB 5152×3864 image/jpeg
#14    DSCN0533.JPG               rd  8744 KB 5152×3864 image/jpeg
#15    DSCN0534.JPG               rd  8678 KB 5152×3864 image/jpeg
…. will show all pictures and videos

 

gphoto2 –get-all-files

Make sure you are in the directory you want to be in to download the files

 

Camara PTP with gphoto2

 


Using Eclipse to remote debug Python on a Raspberry Pi

Overview

Using Eclipse on a workstation to debug Python scripts running on a Raspberry Pi is a very powerful python debugging tool. In this howto post the remote debugging workstation will be Linux, my favorite OS, but this should also work the same if you are using a Windows OS remote workstation. You will need to install Eclipse IDE and Pydev plugin on the workstation. On the Raspberry Pi you can install the Eclipse IDE and Pydev plugin, but running Eclipse on the Pi is painfully show, so I don’t recommend installing Eclipse IDE on the Pi we will only be installing part on the Pydev plugin on the Pi to makes this work.

You might be asking yourself why not install with the package manger….first reason is current apt version is Eclipse Mars 3.8 and we will be installing the latest stable version (at the time on this writing) Eclipse Neon 4.6 or higher. Second reason is Eclipse is very picky about what versions on both Eclipse itself and the plugin’s and I have found it to be much more crash proof if you install the way I’m describing in this how-to. For example you will not be able to use the install wizard or drag and drop to install PyDev (plugin that allows Python debugging) because it needs the 4.x Neon.

Network Connection

Both devices need to communicate over an IP connection this can be a local network LAN or even over the Internet WAN. For this post I’m assuming the Pi and the workstation are on the same LAN network. Over the Internet you need to make sure any firewalls between the two device will pass port 5678.  You will also need to setup port forwarding to the remote workstation. The Pi makes a connection to the workstation when the Python script Pydev debug function pydevd.settrace() is called, more on that later on.

Eclipse Remote Debug

Click to enlarge

 

Check the Java JRE Release number

Eclipse IDE is a java application and the latest as of this post is Eclipse Platform 4.6 Neon. It requires Java 8 (or later) JRE or JDK to run. The installer also uses java, so the first thing you need to do is check and see if you have Java Runtime Environment JRE installed and correct release.

$ java -version
java version “1.8.0_112”
Java(TM) SE Runtime Environment (build 1.8.0_112-b15)
Java HotSpot(TM) 64-Bit Server VM (build 25.112-b15, mixed mode)

On my Linux workstation it shows java version “1.8.0_112” which is Java 8 you look at the second digit of the version to determine the release number, so 1.7 would be Java 7 release. If you don’t have Java release 8 or higher or you get “command not found” you need to figure out why and fix before continuing.

Installing the Eclipse IDE and Pydev plugin on the Remote Workstation

How that you have verified you have Java 8 JRE or higher installed lets download the Eclipse installer. Open your browser and point it to http://www.eclipse.org/downloads/eclipse-packages/ and download the installer. Once you select you system type it will download a tar.gz file “eclipse-inst-linux64.tar.gz” after it downloads unpack it.

$ tar xvfz eclipse-inst-linux64.tar.gz

Change into the “eclipse-installer” sub directory you just unpacked the install to and run the installer.

$ cd eclipse-installer
$ ./eclipse-inst

 

screenshot_eclipse_installer

Click to Enlarge

The Installer will load and you can pick any of the IDE’s that best fits your needs, I chose “Eclipse IDE for PHP Developers”, since I also program in php and includes GIT plugin. After you select one it will then ask you where you want to install to, the default is home directory. Remember or write down where it is install you will need that later to run eclipse.

Click to Enlarge

Click to Enlarge

After it installs chose launch and eclipse should load. To run eclipse later you will find it in the /home/your_user_name/eclipse/php-neon/eclipse for my install /home/billf/eclipse/php-neon/eclipse. To run eclipse from the command line:

$ cd /home/billf/eclipse/php-neon/eclipse
$ ./eclipse

 

Click to Enlarge

Click to Enlarge

 

Installing the Pydev plugin

You need the Pydev plugin to debug and create Python code in Eclipse. There are several ways to install the plugin, but the easiest and fastest is to go to the marketplace.eclipse.com site and use the drag and drop feature. To do that you need to have eclipse up and go to http://marketplace.eclipse.org/ search for Pydev.

 

Click to Enlarge

Click to Enlarge

You can use the Drag and Drop install plugin by dragging the Install button button to your running Eclipse workspace. You may need to minimize the welcome window so the workspace is available should look like this:

Click to Enlarge

Click to Enlarge

Once you drag and drop the install button this window will appear:

Click to Enlarge

Click to Enlarge

Make sure all the boxes are check then hit confirm. It will start the install and stop popping up this window

Click to Enlarge

Click to Enlarge

 

You need to hit “Select All” and then “OK” if you do not hit “Select All” it will terminate the install with no message and you have to reinstall the plugin. It will continue to install the plugin and ask you to restart once it finishes. Select restart and Eclipse will restart and you will have the Pydev plugin installed.

There is one other plugin I recommend using that allows to to ssh into the Pi and browse the file and use Eclipse IDE features to develop and edit existing Python scripts. The install is the same as installing Pydev. The plugin is called “Remote System Explorer” search for in the market place and install.

Click to Enlarge

Click to Enlarge

 

Once you have completed the Eclpise install and plugin installs you will have a IDE that can debug and develop Python code.

Configuring the Raspberry Pi to allow remote debugging

Like I mentioned in the overview you can install Eclipse on the Pi, but it is painfully slow, so I do not recommend. If you want to install and try for yourself just “sudo apt-get install eclipse”  or follow the the same steps you did to install Eclipse and the Pydev plugin from above for the remote workstation and you can do both the remote debugging I describe here and standalone. You can skip the rest of the Pi setup below and go right to testing if you installed Eclipse and PyDev plugin on the Pi.

I have figured out that you can preform remote debugging on the Raspberry Pi by just installing the pysrc files from the PyDev plugin for Eclipse. What this does is allow to you preform remote debugging without installing Eclipse on the Raspberry Pi, so this complete setup can be done via ssh on a headless Pi.  The other plus for only installing what you need is the full Eclipse install is over 200M where just the PyDev pysrc is only about 20M.

Installing PyDev minimum on the Pi

The location for the PyDev source plugin files is http://www.pydev.org/updates/ when you go there you will see it is just a redirect to the latest stable source.

Click to Enlarge

Click to Enlarge

Click on the link and it will take you to the latest stable version then select plugins.

Click to Enlarge

Click to Enlarge

In the plugin source sub-directory select org.python.pydev_5.4.0.201611281236.jar depending on the release the numbers will change. The file normally is the very last one, you need to make sure it starts with org.python.pydev_ that is the jar that has the pysrc source we need to install on the Pi.

Click to Enlarge

Click to Enlarge

Don’t download by clicking on the link we will use wget to download directly on the Pi. The source pysrc we are installing here and the version of the plugin on the workstation do not need to match, so once you install this on the Pi and a new version of PyDev is updated on the remote workstation you don’t need to go back and update the version we are installing on the Pi. Having said that, updating this on the Pi from time to time is a good idea since there may be fixes that would improve performance or security.

How to install on the Pi

To install the necessary files, ssh into the Pi and create a directory in your home directory that we will install the pysrc into. I doesn’t matter where you create it, but you need note where you are installing pysrc it is used in the “sys.path.append()” function in your Python script you will be debugging. Create the directory and change into that directory:

$ mkdir $HOME/eclipse-PyDev
$ cd $HOME/eclipse-PyDev/

 

Next step is to download the source code. To do this right click on the link and select “Copy link location” then type “wget” and paste the source URL.
Note:you need to remove the colon they put in front of org in the link name or else you will get a 404: not found from wget.

$ ~/eclipse-PyDev $ wget http://dl.bintray.com/fabioz/pydev/5.4.0/plugins/org.python.pydev_5.4.0.201611281236.jar

Next step we need to unpack using unzip.

$ ~/eclipse-PyDev $ unzip org.python.pydev_5.4.0.201611281236.jar

If you followed the above step you have a remote workstation that can debug Python scripts using Eclipse IDE.

Optional: You will get a warning on the Pi when you enable remote debugging and you run the script, you need to do the following to fix. Install cython and build command to get rid of this warning message. Doesn’t seem to affect the debugging, so you can skip, but is easy to fix.

warning: Debugger speedups using cython not found. Run ‘”/usr/bin/python” “/home/billf/eclipse-PyDev/pysrc/setup_cython.py” build_ext –inplace’ to build.

Install both cython and cython3 so we can support both 2.x and 3.x Python scrips.

sudo apt-get install cython cython3

Next build the ext’s for setup_cython.py for both Python 2.x and Python 3.x

/usr/bin/python /home/billf/eclipse-PyDev/pysrc/setup_cython.py build_ext –inplace
/usr/bin/python3 /home/billf/eclipse-PyDev/pysrc/setup_cython.py build_ext –inplace

The above steps will get rid of the warning on the Pi when you run a script in remote debug mode.

End Optional:

Testing Eclipse remote debug Python on a Raspberry Pi

 

Here is a example of a simple test script with a RemoteDebug() function you can add and call in any Python scripts you write to do remote debugging works for both Python 2.x and Python 3.x scripts



import sys
import os

# pydev remote debugger function 
def RemoteDebug():

		try:
			sys.path.append("/home/billf/eclipse-PyDev/pysrc/")

			import pydevd
			global pydevd

			# if you want debug to break as soon as the pydevd.sttrace 
			# function below is called set "suspend=True and the debug will start on the next line of 
			# python code after the function call.
			# if you want to go directly to a break point change "supend=False" and 
			# insert "pydevd.settrace()"function where you want the break point.
			# pydevd.settrace('Remote IP address or hostname',port=5678, suspend=True)
			pydevd.settrace('mars', port=5678, suspend=True)
			# stdoutToServer and stderrToServer redirect stdout and stderr to eclipse console
			# pydevd.settrace('192.168.0.32', stdoutToServer=True, stderrToServer=True)

	#===========================================================================
	#   settrace() debug arguments .
	# 
	#   host: the user may specify another host,if the debug server is not in the 
	#	  same machine (default is the localhost)
	# 
	#   stdoutToServer: when this is true, the stdout is passed to the debug server
	# 
	#   stderrToServer: when this is true, the stderr is passed to the debug server
	#	            so that they are printed in its console and not in this process console.
	# 
	#   port: specifies which port to use for communicating with the server
	#	  default port 5678
	# 
	#   suspend: whether a breakpoint should be emulated as soon as this function is called.
	# 
	#   trace_only_current_thread: determines if only the current thread will be traced or 
	#			       all current and future threads will also have the tracing enabled.
	# 
	#   overwrite_prev_trace: if True we'll reset the frame.f_trace of frames which are already 
	#	                  being traced
	# 
	#   patch_multiprocessing: if True we'll patch the functions which create new processes 
	#	                   so that launched processes are debugged.
	#===========================================================================

			 # hack force 'xterm' terminal for pydevd debugging. Else will get terminal error with curses module			 
			os.environ['TERM'] = 'xterm'
			print ("Remote Debug works")

		except ImportError:
			sys.stderr.write("Error: " + 
			   "You must add pysrc path to the above sys.path.append statement or to your PYTHONPATH.\n")
			sys.exit(1)

def main():

		global remote_debug

		remote_debug = True

		# Check to see if we enabled remote debugging
		if remote_debug == True:
			RemoteDebug()

		print ("Goodbye")  

if __name__ == "__main__":
		main()

Using Eclipse is out of the scope of this how-to there are many youtube video’s, but will get you started if you never used it before…..

Couple of things you need to understand are prospectives and debugging basics. Lets first talk about prospectives in Eclipse. A prospective is a view of the workspace and depending on what you are doing the windows in that prospective change. For example when in the debug prospective windows that pertain to debugging will display window for Variables, Debug server, the Python script you will be stepping through and the icons below the menu will change too. You can move the windows around change the size of them delete them, minimize, maximize them. Try playing around with them…the good news is it is very easy to reset them back to the default. In the top menu:

To reset the current prospective back to its default:
Windows -> Prospective – > Reset Prospective

 

To go to a prospective you can use the icons in the upper right next to a box that says “Quick Access” no icons will display there until you go to a prospective at least once. If you know the name of the prospective you can type in in the “Quick Access” box for example “debug” will pop-up as a Prospective. We will need to go to the Prospective “debug” to test our remote debugging, so type debug in the “Quick Access” box and select Prospective – Debug.

Click to Enlarge

Click to Enlarge

To try your first remote debug you need to have the example script from above installed on the Pi. For this test installing it in the home directory is fine. From the workstation you are running Eclipse on open up a shell and ssh into the Pi. Copy and paste the example into a file called “remote_debugger_test.py”. edit line 21 and put in the IP address of the workstation running Eclipse, not the Pi’s IP address.

Edit line 21 of the remote_debugger_test.py change to the IP of the pc running Eclipse debug server, not the PI’s IP:
pydevd.settrace(‘192.168.0.32‘, port=5678, suspend=True)

In Eclipse go to the upper menu and start the Debug Server:

To start the remote Debug server in Eclipse:
PyDev -> Start Debug Server

You will see in the console window “Debug Server at port: 5678.

Now go to the ssh shell session and run the the remote_debugger_test.py example.

python remote_debugger_test.py

You should see the Eclipse debug server connect it should look like this.

Click to Enlarge

Click to Enlarge

Use the icons below the menu to step through the lines of code.

Once you step through the code the debug will terminate. Back on the ssh session for the Pi you will see output from the print statement display.

 

$ python remote_debugger_test.py
Remote Debug works
Goodbye

 

Happy Debugging!!!
Bill….

 


tcpdump-examples

See the list of interfaces on which tcpdump can listen:

tcpdump -D

Listen on interface eth0:

tcpdump -i eth0

Listen on any available interface (cannot be done in promiscuous mode. Requires Linux kernel 2.2 or greater):

tcpdump -i any

Be verbose while capturing packets:

tcpdump -v

Be more verbose while capturing packets:

tcpdump -vv

Be very verbose while capturing packets:

tcpdump -vvv

Be verbose and print the data of each packet in both hex and ASCII, excluding the link level header:

tcpdump -v -X

Be verbose and print the data of each packet in both hex and ASCII, also including the link level header:

tcpdump -v -XX

Be less verbose (than the default) while capturing packets:

tcpdump -q

Limit the capture to 100 packets:

tcpdump -c 100

Record the packet capture to a file called capture.cap:

tcpdump -w capture.cap

Record the packet capture to a file called capture.cap but display on-screen how many packets have been captured in real-time:

tcpdump -v -w capture.cap

Display the packets of a file called capture.cap:

tcpdump -r capture.cap

Display the packets using maximum detail of a file called capture.cap:

tcpdump -vvv -r capture.cap

Display IP addresses and port numbers instead of domain and service names when capturing packets (note: on some systems you need to specify -nn to display port numbers):

tcpdump -n

Capture any packets where the destination host is 192.168.1.1. Display IP addresses and port numbers:

tcpdump -n dst host 192.168.1.1

Capture any packets where the source host is 192.168.1.1. Display IP addresses and port numbers:

tcpdump -n src host 192.168.1.1

Capture any packets where the source or destination host is 192.168.1.1. Display IP addresses and port numbers:

tcpdump -n host 192.168.1.1

Capture any packets where the destination network is 192.168.1.0/24. Display IP addresses and port numbers:

tcpdump -n dst net 192.168.1.0/24

Capture any packets where the source network is 192.168.1.0/24. Display IP addresses and port numbers:

tcpdump -n src net 192.168.1.0/24

Capture any packets where the source or destination network is 192.168.1.0/24. Display IP addresses and port numbers:

tcpdump -n net 192.168.1.0/24

Capture any packets where the destination port is 23. Display IP addresses and port numbers:

tcpdump -n dst port 23

Capture any packets where the destination port is is between 1 and 1023 inclusive. Display IP addresses and port numbers:

tcpdump -n dst portrange 1-1023

Capture only TCP packets where the destination port is is between 1 and 1023 inclusive. Display IP addresses and port numbers:

tcpdump -n tcp dst portrange 1-1023

Capture only UDP packets where the destination port is is between 1 and 1023 inclusive. Display IP addresses and port numbers:

tcpdump -n udp dst portrange 1-1023

Capture any packets with destination IP 192.168.1.1 and destination port 23. Display IP addresses and port numbers:

tcpdump -n “dst host 192.168.1.1 and dst port 23”

Capture any packets with destination IP 192.168.1.1 and destination port 80 or 443. Display IP addresses and port numbers:

tcpdump -n “dst host 192.168.1.1 and (dst port 80 or dst port 443)”

Capture any ICMP packets:

tcpdump -v icmp

Capture any ARP packets:

tcpdump -v arp

Capture either ICMP or ARP packets:

tcpdump -v “icmp or arp”

Capture any packets that are broadcast or multicast:

tcpdump -n “broadcast or multicast”

Capture 500 bytes of data for each packet rather than the default of 68 bytes:

tcpdump -s 500

Capture all bytes of data within the packet:

tcpdump -s 0