(Canon) camera image downloads to a network via a Raspberry Pi
Update 31st December 2020
As described on other pages on this site I use a DSLR camera to take time lapse movies of NLC though a similar setup could be used for other purposes such as Aurora and Meteor Imaging.
Typically I leave the camera running for a couple of hours when imaging NLC and end up with about 240 images and around 500 MBytes of data. The question is then how best to download this data to the Network Attached Storage (NAS) on my home network for review and long term storage. There are two options:
The first method is Ok but a bit fiddly and the CF connectors on the camera and memory card only have a life of 5k insertions. Also it is not that easy to reach the camera and in the long term plan the camera will be mounted outside in a weatherproof enclosure.
In the case of the USB connection the camera is too far from my computer for a USB lead to work so I would have to take a laptop to the camera, plug it in and then download the data to the laptop and hence to my network. Getting the laptop close is difficult and unplugging the USB cable each day would eat into the life of 1500 insertions for a USB 2 connector.
My camera is quite close to a network connection and I investigated if the USB data could be sent over the Cat 5 cable while it was also carrying other network traffic but the products on the market implied they may need a dedicated cable.
I then had a thought, the Raspberry Pi that controls the camera is quite close to the camera and has a spare USB port, could I use this as an interface between the DSLR and my NAS ?
A bit of internet research showed I should be able to mount the camera as a file system on the RasPi and then access it from my Windows network using SAMBA. How did this go ? At best I can claim a partial success in that while I can access the files on the RasPi using SAMBA I have not been able to access the file system on the camera from my Windows desktop via SAMBA.
The major problem I encountered is that I am unable to find where on the RasPi the camera file system is mounted. I have followed several leads on the internet but all have come up blank. However all is not lost the RasPi file manager PCManFM together with gphoto2 seems to mount the camera and the camera file system can be accessed via PCManFM. The image on the right shows the Camera in the places list in the left hand column. You should note that this only appears when the camera is connected to the USB port and switched on so it is dynamically recognized.
If we then open the 'Canon Digital Camera' from the places menu we get the screen shown on the left below. this shows the file system on the camera as 'DCIM' as would be expected and below this are the folders containing the images. The strange thing is the mount point shown in the image as 'gphoto2://[usb:001,008]' (note the second number 008 seems to vary sometimes it is 004, this seems to depend on what is connected to other USB ports and if you access the pi from a terminal connected directly to the RasPi or from a remote connection). I have searched the RasPi file system for this mount point but have not been able to locate it. Note: if you right click on the 'Places' entry for Canon Digital Camera' you have the option to unmount the camera so the file manager seems to think it has been mounted - the question is where and how do you access it directly ?
So where does this leave downloading files to my NAS ?
At present I have to do this in two stages.
First I log into the RasPi and copy the DCIM folder and it's sub-folders to a folder on the Pi in this case called 'Camera Data'. This is quite slow each image takes about 15 to 20 secs so to transfer a nights images takes around 50 to 60 minutes. I suspect this may be a function of the camera speed. I sometimes get a message 'Error nnn Unable to lock the camera' but the transfer seems to still take place. The only difference other than the slow speed to when I access the data from a direct connection from a PC is that when accessed from the PC the camera display shows a flashing 'busy' and the CF card access light is bright red. When accessed via PCManFM the camera display shows as normal so it appears ready to take more images though it is still transferring files. The file access light is also a dim red as though it is only accessing the card part of the time.
The good news is that having copied the data to the RasPi the transfer to the PC is quite fast the copying of 500 MBytes to my NAS takes only a couple of minutes.
I suspect the reason that the camera mount point cannot be found is that the file manager in conjunction with gphoto2 have hijacked the USB device mounting and mounted the camera into an 'hidden' place.
If you have any solutions that will fix the speed the camera downloads or where the camera is mounted so I can access it directly from my PC using SAMBA please let me know via the contact details on my home page.
May 2016 Update: A cure for the download problems ?
According to the web page on building a RaspberryPi based Tiny Backup device it should be possible to download all the files from a camera connected via USB using the command 'gphoto2 --get-all-files' from a terminal window however this resulted in an error message 'gphoto2 error (-60 'could not lock the device')
Another web search for the reason for this can across a page on the RaspberryPi forum here that discusses this error. There are several suggestions but including removing and compiling a newer version of GPhoto2, however I have found that it is possible to remove the error message by renaming the following 4 files as shown in the post by AMikeM on 4th Jan 2015 (near the end of the page).
I renamed the following files (by simply
adding an arbitrary prefix to the filename):
Having renamed the files - I just added suspended as a prefix to the names and re-booted my RasPi I can now download the image files from my camera using the command ' 'gphoto2 --get-all-files'.
Note: There is no need to compile and install the latest version of GPhoto2 as suggested on this page - renaming the 4 files to prevent them running solves the problem
The next stage is to investigate using a script to rename the files as per the method in the RaspberryPi based Tiny Backup device above, this will get over the problem that after taking 9999 images the file names start again from img_0001.jpg so I have several different files of the same name on my computer drives.
Note: I have discovered that the tool exiftool can be used to rename files to the time and date they were taken using the following command exiftool *.JPG -d %Y%m%d-%H%M%S-%%c.%%e "-FileName<DateTimeOriginal" I now use this after I have downloaded the files to change the filename. It is then very easy to locate the images from a particular observing session. The only disadvantage with this command is that if two images are taken with the same time to the second the second file will have '(1)' appended to it as the exposure times are only recorded in one second intervals in the EXIF data by the camera.
It should be possible to run the download scrip automatically either when the Python script that controls the camera finishes or else at a specified time.
I also need to decide how to delete the files that I have downloaded from the camera. Do I want to do this automatically and risk loss of images if the files have not been downloaded or do I want to delete them manually each day ? At present I reformat the CF card manually from the camera menu as the quickest way to clear the images.
Update June 2017
When you are downloading your images from the camera using 'gphoto2 --get-all-files' from a terminal window they will by default be downloaded into the folder you run the command from. It makes life easier if you create a new folder first and then cd to this rather than have to move the files afterwards.
the command 'gphoto2 --get-all-files' does what it says it downloads all the files from the camera even if you have downloaded them previously. Gphoto2 has options to download by a RANGE or NAME see the reference manual at http://gphoto.org/doc/manual/ref-gphoto2-cli.html . When recording NLC files they may go over the date change boundary (at midnight local) so to only download the latest files requires a range with both date and time.
The definition of RANGE near the top of the reference manual seems to give each file a number - it is not clear how this relates if at all to the camera file numbering. At present I am downloading all the files and then deleting them from the camera.
Gphoto2 has the option of --new which should download only new files but this depends on both the camera and driver supporting it. Worth a try
Deleting files can be done either via the camera menu or using Gphoto2 with the options -D and -R to set recursion - beware of running this if you have any files you want on the camera - it is the same as formatting the memory card.
It might be possible to use the Linux file manager to download the files but this requires Gphoto2 relinquishing it's lock on the camera port.
Update May 2017
Downloading the large number of image files from the camera to the Raspberry Pi takes a considerable length of time particularly if you take large RAW images. This appears to be a function of the speed of the camera I am using - a Canon 10D rather than the Raspberry Pi or the USB connection.
To reduce the time wasted by this I have modified the Python Script to download all the images on the camera to the Raspberry Pi at the end of the imaging session. I have not included commands to delete the files from the camera as there is a risk that the images will be lost if the session terminates abnormally. I tried downloading only the new files but it appears there is some problem with the support for this on the Canon 10D though it works with the Canon supplied software.
If you don't want the images downloaded automatically the best thing to do is just 'comment out' the lines in the script that download the images. The revised script is available on this link.
© 2021 John Murrell
Clicking on the Home logo will return you to my home page