2009-12-27
Update on Sansa Fuze
Just to quickly update on the developments regarding the Sansa Fuze in the last year. Most importantly, the problems with video have been pretty much solved (at least for me) with video4fuze. Secondly, new firmware update has improved id tag handling significantly, among other things. There is even newer firmware available, but it's buggy and offers no real improvements, so I'm skipping it.
2008-12-26
More on Sansa Fuze's video format
Sansa Fuze MP3 player only supports a very specific version of the AVI file format for it's video output. This format is not documented anywhere and can only be, as far as I know, produced with Sansa Media Converter (SMC) Windows program. I have spent some time familiarising myself with the video files produced by SMC. This blog post was written to document some of the aspects I have learnt so far.
I hope at least someone finds some interesting information here and perhaps we can come closer to a working video conversion. This is still very much a Work in Progress. I intend to update this entry if any new information pops up. I'm also happy to answer any questions I can in the comment section. See also my previous entry on this subject.
AVI files
AVI files consist of chunks. For a recent general discussion of AVI files, see this pdf file. If you use Python, AVI chunks can be read, for example, with Python's chunk module. See Python's reference page for a nice explanation of chunks in general.
AVI file is actually a RIFF file. A standard RIFF files have 3 types of chunks. First comes the "RIFF Form Header". Then there are normal chunks and list chunks.
A chunk has a 4 byte identifier, a 4 byte chunk size, followed by the actual data. A list chunk has identifier 'LIST', a 4 byte chunk size, a 4 byte list identifier, followed by data. RIFF Form header has identifier 'RIFF', a 4 byte file size, a 4 byte form identifier (in this case, 'AVI '), followed by the actual data, in this case the other chunks.
File Structure
Fuze's video format is, in essence, a standard AVI, but the requirements for a working file are specific. I analysed a very short example file generated with SMC. In this example file, the file structure was as follows:
Some observations
Here are a few random observations about Sansa Fuze's video files.
Further information
A few sources of information about AVI files in general:
I hope at least someone finds some interesting information here and perhaps we can come closer to a working video conversion. This is still very much a Work in Progress. I intend to update this entry if any new information pops up. I'm also happy to answer any questions I can in the comment section. See also my previous entry on this subject.
AVI files
AVI files consist of chunks. For a recent general discussion of AVI files, see this pdf file. If you use Python, AVI chunks can be read, for example, with Python's chunk module. See Python's reference page for a nice explanation of chunks in general.
AVI file is actually a RIFF file. A standard RIFF files have 3 types of chunks. First comes the "RIFF Form Header". Then there are normal chunks and list chunks.
A chunk has a 4 byte identifier, a 4 byte chunk size, followed by the actual data. A list chunk has identifier 'LIST', a 4 byte chunk size, a 4 byte list identifier, followed by data. RIFF Form header has identifier 'RIFF', a 4 byte file size, a 4 byte form identifier (in this case, 'AVI '), followed by the actual data, in this case the other chunks.
File Structure
Fuze's video format is, in essence, a standard AVI, but the requirements for a working file are specific. I analysed a very short example file generated with SMC. In this example file, the file structure was as follows:
- 'RIFF' form header
- 'hdrl' LIST chunk, header information
- 'avih' chunk (56), avi file headers
- 'strl' LIST chunk, which describes the video stream
- 'strh' chunk, stream header
- 'strf' chunk, further stream description
- 'indx' chunk, odml stream indexing
- 'strl' LIST chunk, which describes the audio stream
- 'strh' chunk, stream header
- 'strf' chunk, further stream description
- 'indx' chunk, odml stream indexing
- 'JUNK' chunk, which is just filler
- 'odml' LIST chunk, OpenDML header list
- 'dmlh' chunk (248), OpenDML header chunk
- 'INFO' LIST chunk
- In this case, this list chunk has just one subchunk called 'ISFT' (software), with the string 'InterVideo\x00\x00'.
- 'movi' LIST chunk, containing the actual video and audio data as subchunks
- In our example, this list chunk has just 4 subchunks called, ix00, ix01, 00dc, 01wb. The first two are odml index chunks, the third one is video, the last one is audio.
- 'idx1' chunk. Legacy index chunk that is actually not required by Sansa Fuze.
Some observations
Here are a few random observations about Sansa Fuze's video files.
- Wrong ulFlags in the avih header do not affect playback. It seems that the player just ignores this and in fact much of the headers.
- idx1 chunk is not required. Apparently it is not even read before playback reaches the end.
- Fuze uses OpenDML (AVI2) indexing. 'odml' chunk is absolutely required. So are 'indx' subchunks under 'strl' chunks, and ix## subchunks under movi chunk. Mencoder unfortunately only creates odml compatible files if file size is greater than 1 GB. I have tried modifying the sources so that odml index is created every time, but these files do not seem to work any better. It is probable that Fuze doesn't like Mencoder's method of starting a new RIFF chunk whenever odml index is used. Perhaps there is another encoder somewhere that can create the odml chunk every time. I understand that Windows program VirtualDubMod creates OpenDML compatible files, but I haven't had the time to find out how well it works.
Further information
A few sources of information about AVI files in general:
- A wiki page about AVI.
- Example code in C.
- Python module for reading/writing binary data.
- AVI in general.
2008-12-06
HP Laserjet 1020 and Casio EX-Z80 on Ubuntu 8.10
A few notes on two devices I tried today.
The printer, HP LaserJet 1020, was a bit odd. Once I plugged it in, Ubuntu 8.10 (Intrepid Ibex) seemed to recognize it and installed drivers. But when I sent a test page it was never printed. There were no error messages or anything, though. This thread on Ubuntu Forums helped. The relevant part:
The camera, Casio EX-Z80, worked without problems. It was instantly recognised by Ubuntu and data transfer worked nicely.
The printer, HP LaserJet 1020, was a bit odd. Once I plugged it in, Ubuntu 8.10 (Intrepid Ibex) seemed to recognize it and installed drivers. But when I sent a test page it was never printed. There were no error messages or anything, though. This thread on Ubuntu Forums helped. The relevant part:
- sudo apt-get install hplip-gui
- Delete earlier printer setup in System->Administration->Printing
- sudo hp-setup, next->next->next
- System->Administration->Printing, right-click on the printer icon, make sure that it's enabled
The camera, Casio EX-Z80, worked without problems. It was instantly recognised by Ubuntu and data transfer worked nicely.
2008-10-11
New Sansa Fuze firmware with Ogg support
There is a new firmware update available for Sansa Fuze. It updates the firmware to version 01.01.15. Most notable improvement, long awaited one even, is support for ogg and flac files. The first ogg file I tried froze up my player and I had to reboot it, but after that I haven't encountered problems. Not yet at least.
Firmware update on Linux was easy following the instructions provided and involved just unpacking the new firmware file to the root of the device. And using the Sansa Fuze is easier now on Ubuntu also because the new Ubuntu 8.10 Beta seems to recognize the player without any command line magic required previously.
Playlist seem to work somewhat better too. Now if only the huge problems with video conversion were fixed. Perhaps after that I could actually recommend this player to others.
Firmware update on Linux was easy following the instructions provided and involved just unpacking the new firmware file to the root of the device. And using the Sansa Fuze is easier now on Ubuntu also because the new Ubuntu 8.10 Beta seems to recognize the player without any command line magic required previously.
Playlist seem to work somewhat better too. Now if only the huge problems with video conversion were fixed. Perhaps after that I could actually recommend this player to others.
Ubuntu 8.10 (Beta)
I had some extra time so I decided to upgrade my Ubuntu 8.04 to Ubuntu 8.10 Beta (codenamed Intrepid Ibex, would you believe it). My earlier pursuits with Ubuntu and my Acer 2920Z are detailed here for 7.10 and here for 8.04.
My first try to upgrade failed miserably. The reason was probably that I had selected a country specific server from program sources. I switched to the main server and tried again. This time everything seemed to go fine, but apparently it did not, as Gnome wouldn't load after reboot. My first try had probably messed something up. So, from console, I used apt-get to redo parts of the upgrade (I tried several commands, such as sudo apt-get update, upgrade, dist-upgrade, install, autoclean). After this Gnome loaded and everything seemed to be fine.
Everything, that is, except wlan. Madwifi drivers wouldn't work anymore. I had a look at the madwifi ticket for this chipset and learned that it was no longer necessary to use the hack I had been using earlier. The patch had been integrated to a madwifi branch. Unfortunately, this branch was not yet available from Ubuntu repositories, so it had to be installed manually. So I downloaded the new madwifi HAL, compiled and installed it (make, make install). After reboot wlan was working again.
It seems that after the upgrade the random problems with sound seem to be gone and, finally, the internal microphones are working. Incredibly, even TV-out works now. It was a struggle finding the correct settings for the displays, though. When mirroring screens the main laptop display was forced into a suboptimal resolution. Through trial and error, reboots, and turning mirroring off, I finally found a configuration which enabled me to watch video clips on my TV while surfing the net.
So, again, some minor problems with the upgrade, but nothing major. I'll update this entry if anything interesting happens.
My first try to upgrade failed miserably. The reason was probably that I had selected a country specific server from program sources. I switched to the main server and tried again. This time everything seemed to go fine, but apparently it did not, as Gnome wouldn't load after reboot. My first try had probably messed something up. So, from console, I used apt-get to redo parts of the upgrade (I tried several commands, such as sudo apt-get update, upgrade, dist-upgrade, install, autoclean). After this Gnome loaded and everything seemed to be fine.
Everything, that is, except wlan. Madwifi drivers wouldn't work anymore. I had a look at the madwifi ticket for this chipset and learned that it was no longer necessary to use the hack I had been using earlier. The patch had been integrated to a madwifi branch. Unfortunately, this branch was not yet available from Ubuntu repositories, so it had to be installed manually. So I downloaded the new madwifi HAL, compiled and installed it (make, make install). After reboot wlan was working again.
It seems that after the upgrade the random problems with sound seem to be gone and, finally, the internal microphones are working. Incredibly, even TV-out works now. It was a struggle finding the correct settings for the displays, though. When mirroring screens the main laptop display was forced into a suboptimal resolution. Through trial and error, reboots, and turning mirroring off, I finally found a configuration which enabled me to watch video clips on my TV while surfing the net.
So, again, some minor problems with the upgrade, but nothing major. I'll update this entry if anything interesting happens.
2008-08-02
Video on SanDisk Sansa Fuze
I recently wrote a post about my SanDisk Sansa Fuze mp3 player, especially on getting it to work on Linux. This post is about a specific issue with it, namely video. (Executive summary: it doesn't work.)
The Problem
Getting video working on the Sansa is ridiculously difficult if you are not able to use their archaic Sansa Media Converter Windows software, regardless of your own OS. The Sansa Fuze is in theory a pretty nice media player, which (again in theory) can play mpeg videos in an avi container. In practise, the Sansa expects this video to be in a absurdly specific format. For everything else it displays an error message. To get video into the correct format you must use the converter they have provided.
Sansa Media Converter
Sansa Media Converter is a horrible little video converting program, a true nightmare to use, and the only supported way of getting video to play on the player. It seems to be a rebranded Intervision product, which converts some common video formats to the format used by the Sansa Fuze. It is naturally Windows only and even on Windows it requires the player to be plugged in for conversion to be possible. It doesn't even load on Wine. On a virtual Windows machine it starts but refuses to convert as it can't see the player even if it's plugged in. Also on my old Windows machine it refuses to see the player. Sansa Media Converter represents everything that is wrong in today's computing, and does that faithfully. The pox on the people who wrote it.
Working Example
On the Sansa there is a preloaded video file, which can be used to investigate the format accepted by the player. Also, I have converted some other examples with the Sansa converter on a machine it actually accepted. I have loaded the files into a Windows program abcAvi Tag Editor (works under Wine), which displays a lot of information on avi files. Here's what it has to say about the preloaded example:
My attempts
I attempted to reproduce a working file with mencoder. I used the following command on the example file. The command does not produce a file acceptable to Sansa.
The command does nothing to the audio and video streams, it just copies them to a new file. Only thing that changes is the container part of the file. I manually set the resolution, frame rate and FourCC code for this file. (The noidx option is actually not actually needed, as the example file has an index. That's just something I forgot there.) This resulted in a file described below
There are some notable differences to the working example file. I suspect the most important one is "AVIF_ISINTERLEAVED" flag, which means that this avi file has interleaved video and audio. It is possible that this may be too much for the Sansa Fuze to handle.
Mencoder/Mplayer has, by the way, an option called -ni, which forces non-interleaved mode for reading files. Unfortunately it doesn's seem to affect decoding.
It is worth mentioning that the Sansa isn't picky about every little detail. I changed the FourCC code, for instance, on the preloaded clip to some false value and the clip still played.
VirtualDub
There is this Windows program called VirtualDub, which has an option for turning of interleaving avi files. I tried it under Wine. The program worked, but unfortunately, the file it produced had other differences to the examples. I haven't found options to fix those differences in the program
Possible Solutions
I'm no expert when it comes to video encoding, so currently I have no ideas how to move forward. I guess there might be some program somewhere that can reproduce Sansa's format, or perhaps it could be possible to make a non-interleaved version of avi codecs and try those.
For reference you might want to see further information on the video clip and this discussion.
Any help with this would be appreciated.
The Problem
Getting video working on the Sansa is ridiculously difficult if you are not able to use their archaic Sansa Media Converter Windows software, regardless of your own OS. The Sansa Fuze is in theory a pretty nice media player, which (again in theory) can play mpeg videos in an avi container. In practise, the Sansa expects this video to be in a absurdly specific format. For everything else it displays an error message. To get video into the correct format you must use the converter they have provided.
Sansa Media Converter
Sansa Media Converter is a horrible little video converting program, a true nightmare to use, and the only supported way of getting video to play on the player. It seems to be a rebranded Intervision product, which converts some common video formats to the format used by the Sansa Fuze. It is naturally Windows only and even on Windows it requires the player to be plugged in for conversion to be possible. It doesn't even load on Wine. On a virtual Windows machine it starts but refuses to convert as it can't see the player even if it's plugged in. Also on my old Windows machine it refuses to see the player. Sansa Media Converter represents everything that is wrong in today's computing, and does that faithfully. The pox on the people who wrote it.
Working Example
On the Sansa there is a preloaded video file, which can be used to investigate the format accepted by the player. Also, I have converted some other examples with the Sansa converter on a machine it actually accepted. I have loaded the files into a Windows program abcAvi Tag Editor (works under Wine), which displays a lot of information on avi files. Here's what it has to say about the preloaded example:
[RIFF AVI 20988512]: Audio/Video Interleaved File
[LIST hdrl 64638]: Format Definition Chunk
[avih 56]: Main AVI Header
[dwMicroSecPerFrame]: Microseconds Per Frame: 50000
[dwMaxBytesPerSec]: Maximum Bytes Per Second: 101885
[dwPaddingGranularity]: Padding Granularity (bytes): 1
[dwFlags]: Flags 0x00000810
[AVIF_HASINDEX]: AVI file has 'idx1' chunk
[AVIF_TRUSTCKTYPE]: use chunk type to find key frames
[dwTotalFrames]: Number Of Frames: 4137
[dwInitialFrames]: Numer Of Initial Frames: 0
[dwStreams]: Number Of Streams: 2
[dwSuggestedBufferSize]: Suggested Buffer Size (bytes): 15237
[dwWidth]: Image Width (pixels): 224
[dwHeight]: Image Height (pixels): 176
[LIST strl 32148]: Stream Format Definition Chunk
[strh 56]: Stream Header
[fccType]: Type: vids (Video Data)
[fccHandler]: Handler: 0x00000000 (No Compression <0x00000000>)
[dwFlags]: Flags 0x00000000
[wPriority]: Stream Priority: 0
[wLanguage]: Language: 0 (Undefined)
[dwInitialFrames]: Numer Of Initial Frames: 0
[dwScale]: Stream Scale: 500000
[dwRate]: Stream Rate: 10000000
[dwStart]: Stream Start: 0
[dwLength]: Stream Duration: 4137
[dwSuggestedBufferSize]: Suggested Buffer Size (bytes): 15237
[dwQuality]: Stream Quality (1-10000): 0
[dwSampleSize]: Sample Size: 0
[rcFrame]: Frame Size Of Stream
[wX1]: Top: 0
[wY1]: Bottom: 0
[wX2]: Left: 0
[wY2]: Right: 0
[strf 40]: Video Stream Format
[biSize]: Header Size (bytes): 40
[biWidth]: Image Width (pixels): 224
[biHeight]: Image Height (pixels): 176
[biPlanes]: Number Of Planes: 1
[biBitCount]: Number Of Bits Per Pixel: 32
[biCompression]: Compression Algorithm: DX50 (DivX 5.0 codec 'DX50' <0x30355844>)
[biSizeImage]: Image Size (bytes, 0 = uncompressed): 230400
[biXPelsPerMeter]: Horizontal resolution of target device: 0
[biYPelsPerMeter]: Vertical resolution of target device: 0
[biClrUsed]: Number of colors used (0 = maximum): 0
[biClrImportant]: Number of important colors (0 = all): 0
[indx 32024]: AVI2 (OpenDML) Super Index Chunk
[LIST strl 32138]: Stream Format Definition Chunk
[strh 56]: Stream Header
[fccType]: Type: auds (Audio Data)
[fccHandler]: Handler: 0x00000000 (No Compression <0x00000000>)
[dwFlags]: Flags 0x00000000
[wPriority]: Stream Priority: 0
[wLanguage]: Language: 0 (Undefined)
[dwInitialFrames]: Numer Of Initial Frames: 0
[dwScale]: Stream Scale: 1
[dwRate]: Stream Rate: 16000
[dwStart]: Stream Start: 417
[dwLength]: Stream Duration: 3309595
[dwSuggestedBufferSize]: Suggested Buffer Size (bytes): 2008
[dwQuality]: Stream Quality (1-10000): 0
[dwSampleSize]: Sample Size: 1
[rcFrame]: Frame Size Of Stream
[wX1]: Top: 0
[wY1]: Bottom: 0
[wX2]: Left: 0
[wY2]: Right: 0
[strf 30]: Audio Stream Format
[wFormatTag]: Audio Format: 0x0055 (MPEG-1 Layer 3 (MP3) <0x0055>)
[nChannels]: Number of channels: 2
[nSamplesPerSec]: Frequency of the sample rate (Hz): 44100
[nAvgBytesPerSec]: Average data rate (byte/s): 16000
[nBlockAlign]: Block alignment of the data (bytes): 1
[wBitsPerSample]: Number of bits per sample (0 = undefined): 0
[cbSize]: Size of extra information (bytes): 12
[wID]: Type: 0x0002 (Constant frame size)
[fwFlags]: Flags: 0x0000 (ISO Padding)
[nBlockSize]: Block size (bytes): 417
[nFramesPerBlock]: Number of frames per block: 1
[nCodecDelay]: Codec delay (ms): 0
[indx 32024]: AVI2 (OpenDML) Super Index Chunk
[LIST odml 260]: AVI2 (OpenDML) Format Definition Chunk
[dmlh 248]: Total Number Of Frames AVI2 (OpenDML) 4137
[JUNK 34]: Padding
[LIST movi 20824326]: Audio/Video Data
Skipped
[idx1 99312]: AVI Index Chunk
[LIST INFO 92]: RIFF Textual Informative Details
[ILNG 10]: Language: Undefined
[IRTD 8]: Rating: Unrated
[INAM 12]: Title: sansatesti
[IPRD 6]: Product: Movie
[ISFT 12]: Software Used: IntraAudio
[LIST MID 58]: Movie IDentifier Tags
[TITL 12]: Title: sansatesti
[AGES 8]: Rating: Unrated
[LANG 10]: Language: Undefined
[IDVX 128]: Unknown Tag
[LIST DIVXTAG 128]: IDivX tags
[Movie 32]: Title: sansatest
[Author 28]: Director:
[Year 4]: Creation Date Of Subject:
[Comment 48]: Comments:
[Genre 00000003]: IDivX Genre: 017 (Movie)
[Rated 00000001]: IDivX Rating: 0 (Unrated)
My attempts
I attempted to reproduce a working file with mencoder. I used the following command on the example file. The command does not produce a file acceptable to Sansa.
mencoder -ofps 20 -vf scale=224:176 -ni -noidx -nocache -ovc copy -oac copy -ffourcc DX50 -of avi infile.avi -o outfile.avi
The command does nothing to the audio and video streams, it just copies them to a new file. Only thing that changes is the container part of the file. I manually set the resolution, frame rate and FourCC code for this file. (The noidx option is actually not actually needed, as the example file has an index. That's just something I forgot there.) This resulted in a file described below
[RIFF AVI 20719086]: Audio/Video Interleaved File
[LIST hdrl 306]: Format Definition Chunk
[avih 56]: Main AVI Header
[dwMicroSecPerFrame]: Microseconds Per Frame: 50000
[dwMaxBytesPerSec]: Maximum Bytes Per Second: 0
[dwPaddingGranularity]: Padding Granularity (bytes): 0
[dwFlags]: Flags 0x00000900
[AVIF_ISINTERLEAVED]: AVI file is interleaved
[AVIF_TRUSTCKTYPE]: use chunk type to find key frames
[dwTotalFrames]: Number Of Frames: 0
[dwInitialFrames]: Numer Of Initial Frames: 0
[dwStreams]: Number Of Streams: 2
[dwSuggestedBufferSize]: Suggested Buffer Size (bytes): 0
[dwWidth]: Image Width (pixels): 224
[dwHeight]: Image Height (pixels): 176
[LIST strl 116]: Stream Format Definition Chunk
[strh 56]: Stream Header
[fccType]: Type: vids (Video Data)
[fccHandler]: Handler: DX50 (DivX 5.0 codec 'DX50' <0x30355844>)
[dwFlags]: Flags 0x00000000
[wPriority]: Stream Priority: 0
[wLanguage]: Language: 0 (Undefined)
[dwInitialFrames]: Numer Of Initial Frames: 0
[dwScale]: Stream Scale: 1
[dwRate]: Stream Rate: 20
[dwStart]: Stream Start: 0
[dwLength]: Stream Duration: 4137
[dwSuggestedBufferSize]: Suggested Buffer Size (bytes): 15229
[dwQuality]: Stream Quality (1-10000): 0
[dwSampleSize]: Sample Size: 0
[rcFrame]: Frame Size Of Stream
[wX1]: Top: 0
[wY1]: Bottom: 0
[wX2]: Left: 224
[wY2]: Right: 176
[strf 40]: Video Stream Format
[biSize]: Header Size (bytes): 40
[biWidth]: Image Width (pixels): 224
[biHeight]: Image Height (pixels): 176
[biPlanes]: Number Of Planes: 1
[biBitCount]: Number Of Bits Per Pixel: 32
[biCompression]: Compression Algorithm: DX50 (DivX 5.0 codec 'DX50' <0x30355844>)
[biSizeImage]: Image Size (bytes, 0 = uncompressed): 230400
[biXPelsPerMeter]: Horizontal resolution of target device: 0
[biYPelsPerMeter]: Vertical resolution of target device: 0
[biClrUsed]: Number of colors used (0 = maximum): 0
[biClrImportant]: Number of important colors (0 = all): 0
[LIST strl 106]: Stream Format Definition Chunk
[strh 56]: Stream Header
[fccType]: Type: auds (Audio Data)
[fccHandler]: Handler: U (Unknown Four Character Code 'U' <0x00000055>)
[dwFlags]: Flags 0x00000000
[wPriority]: Stream Priority: 0
[wLanguage]: Language: 0 (Undefined)
[dwInitialFrames]: Numer Of Initial Frames: 0
[dwScale]: Stream Scale: 1
[dwRate]: Stream Rate: 16000
[dwStart]: Stream Start: 417
[dwLength]: Stream Duration: 3309595
[dwSuggestedBufferSize]: Suggested Buffer Size (bytes): 8000
[dwQuality]: Stream Quality (1-10000): 0
[dwSampleSize]: Sample Size: 1
[rcFrame]: Frame Size Of Stream
[wX1]: Top: 0
[wY1]: Bottom: 0
[wX2]: Left: 0
[wY2]: Right: 0
[strf 30]: Audio Stream Format
[wFormatTag]: Audio Format: 0x0055 (MPEG-1 Layer 3 (MP3) <0x0055>)
[nChannels]: Number of channels: 2
[nSamplesPerSec]: Frequency of the sample rate (Hz): 44100
[nAvgBytesPerSec]: Average data rate (byte/s): 16000
[nBlockAlign]: Block alignment of the data (bytes): 1
[wBitsPerSample]: Number of bits per sample (0 = undefined): 0
[cbSize]: Size of extra information (bytes): 12
[wID]: Type: 0x0002 (Constant frame size)
[fwFlags]: Flags: 0x0000 (ISO Padding)
[nBlockSize]: Block size (bytes): 417
[nFramesPerBlock]: Number of frames per block: 1
[nCodecDelay]: Codec delay (ms): 0
[LIST INFO 42]: RIFF Textual Informative Details
[ISFT 30]: Software Used: MEncoder 2:1.0~rc2-0ubuntu13
[JUNK 3712]: Padding
[LIST movi 20714990]: Audio/Video Data
Skipped
There are some notable differences to the working example file. I suspect the most important one is "AVIF_ISINTERLEAVED" flag, which means that this avi file has interleaved video and audio. It is possible that this may be too much for the Sansa Fuze to handle.
Mencoder/Mplayer has, by the way, an option called -ni, which forces non-interleaved mode for reading files. Unfortunately it doesn's seem to affect decoding.
It is worth mentioning that the Sansa isn't picky about every little detail. I changed the FourCC code, for instance, on the preloaded clip to some false value and the clip still played.
VirtualDub
There is this Windows program called VirtualDub, which has an option for turning of interleaving avi files. I tried it under Wine. The program worked, but unfortunately, the file it produced had other differences to the examples. I haven't found options to fix those differences in the program
Possible Solutions
I'm no expert when it comes to video encoding, so currently I have no ideas how to move forward. I guess there might be some program somewhere that can reproduce Sansa's format, or perhaps it could be possible to make a non-interleaved version of avi codecs and try those.
For reference you might want to see further information on the video clip and this discussion.
Any help with this would be appreciated.
2008-08-01
SanDisk Sansa Fuze 8GB on Ubuntu 8.04
(Update: see also my posts about the video problems and about the new firmware with Ogg Vorbis support.)
I've recently purchased a SanDisk Sansa Fuze MP3 player. My old player has served me well for a very long time, but it's age is showing and I thought it was finally time to retire it. These are my notes on getting the Sansa Fuze working with my Acer laptop and Ubuntu Linux.
I selected SanDisk player after googling revealed encouraging reports about it's Linux compatibility and, especially, after I discovered that SanDisk explicitly mentions Linux in it's product specs. It also had some very nice features when compared to some of it's competitors, FM radio and voice recording for example. Also, I had seen some rumours about upcoming Ogg Vorbis support. Now, rumours are rumours and I believe when I see it, but ogg support would be absolutely fantastic.
Connecting
MP3 players in general use two different protocols to communicate with the computer. The older MSC protocol tends to work well with different operating system as the device shows up as a mass storage drive for the computer. Microsoft's newer MTP protocol, designed to implement DRM restrictions to players, usually requires special software. Then, of course, there are some variations of the two. For example, Microsoft apparently uses protocol called MTPZ for Zunes for some reason. Go figure.
Anyway, Sansa Fuze claims to support both of the aforementioned protocols, but only MSC for OSX and Linux. So I wanted to try that protocol first. I connected my player to my Acer 2970Z's USB port and waited. Nothing happened on the computer. The Sansa Fuze claimed it was "connected", but lsusb didn't show anything new. Oh well, I expected this much. So I tried this tip for another SanDisk player (removing kernel USB 2.0 driver with "sudo rmmod ehci-hcd", see also this post) and sure enough the player mounted immediately. At least now I knew my player could be mounted, even if at suboptimal speed. Fortunately I found out that the speed could be improved easily by reloading the kernel module after Fuze has been mounted ("sudo modprobe ehci-hcd"). This remounts the player and copying is much faster. Later I also discovered that if I use a passive USB hub the player seems to mount with no mod-magic from time to time, although not consistently.
Here's the command I use to get the player recognised. The sleep command gives the player some time to mount and increases the chances of this working slightly, I find:
(Update: apparently, it at least sometimes suffices to use "lsusb".)
I copied a few mp3's and an ogg file to the root folder of the player. Mp3's worked, the ogg didn't (as expected, of course). I then copied much of my music library to the device. After doing this I discovered that m3u playlists showed up on the player, but they didn't work. More about this below.
MTP
Unfortunately, MSC mode does not show the content that had been preloaded to the player. So next, I tested the MTP support with MTPFS, which is supposed to enable drag'n'drop copy to MTP players from Linux. Apparently, it also has playlist support. So I disconnected the player, typed sudo apt-get install mtpfs, and set the player to MTP mode. I also reloaded the USB 2 kernel module just to see if it matters. Then I reconnected player. Nothing happened.
Mtpfs actually uses libmtp to connect to the player, so next I installed libmtp's mtp-tools with "sudo apt-get install mtp-tools" and ran mtp-detect, which promptly informed me that "No Devices have been found".
I proceeded to install the latest libmtp and mtp-tools from source (version 0.3.0). I found a helpful post at the Sandisk forums and followed it as far as it was applicable. The installation required some tweaking, as it apparently installed some files to non-standard locations and this left me for example with two versions of tool executables. Now I finally got mtp-detect to see my device. Amarok, Rhythmbox or mtpfs didn't see it, though. But I was still able to download the preloaded content from the player with mtp-tools.
Here is the one-liner I used to generate a command line for getting all the MTP mode tracks from the player:
Not very elegant, very much ad hoc, but worked for me. As the command actually just prints a download command, which you can then check before pasting and running it, there should be no risk trying it. But, of course, I promise nothing and give no guarantees.
At this point I gave up on MTP for the moment and turned the player back to MSC mode. (Information from mtp-detect should probably be posted at libmtp bug tracker. I haven't got around doing this yet, unfortunately, as MTP isn't a must for this device. Perhaps someone else will find the time.)
Video
Getting video playback is a whole different story. Others have tried with no real success (see e.g. reqeya's post). First I copied the video file preloaded to the player and transferred it back with a new name. It worked. Then I tried copying some random avi files to the player. The player detected these files as video, but complained that they were in a wrong format. No surprise there.
Next I downloaded Windows software Sansa Media Converter from Sansa web site and tried it under Wine and in a virtual machine. Neither worked.
I tried numerous different things to get video working. Again and again, the player said that the media format is unsupported. I'm not very optimistic at the moment, as further investigation revealed that the player requires very specific format for the avi container. See the post I wrote about this for further details on the problem and the format used by the player.
I don't know much about video encoding and would appreciate if anyone else could give some advice on getting video work. For reference you might want to see further information on the video clip and this discussion. It's also worth noting that in the past others have had some success with earlier Sansa models and video, but the player's video format has been ridiculously specific. And they have of course changed the format for this model.
Let me know if you find anything useful.
Playlists
I already mentioned that m3u playlists seemed to be supported in principle. Some additional work is required, however, to get them fully functional. I tried creating a simple m3u file in the root folder of the device with only two entries. I created two versions of the file: normal one with a text editor and a dos version with unix2dos. Both showed up on the device but neither worked.
After some additional frustration I installed Easytag from Synaptic. This is how I got my first working playlist with a single song. I then copied this playlist to a different name, opened it in nano, changed the single entry in the file into a different file and saved. This also worked. Then I created my own list with identical input with nano and ran the file through unix2dos. After several tries I got this working too. As far as I know it is essential that the playlist has CR's in the end of every line:
The playlist must also use relative paths and Windows-style backslashes as directory separators.
Quite bizarrely, some of the stuff I tried seemed to work on and off. It seems that there were some random errors, possibly due to me yanking the usb cable on and off constantly. So, to be safe, you probably should wait for a while before detaching the cable, or you should unmount the device cleanly.
I have written some custom scripts that generate a few playlists automatically from my music collection. Unfortunately I noticed that Fuze read these very slowly and worked with them only after pushing menu button and back after the list had been loaded. Strange.
Slooow
The most notable annoyance with the player has been it's slow functioning. Apparently several gigabytes of music on a VFAT drive is just too much for it.
Especially slow is the "database update" after file transfer. This is mostly due to the numerous media files on the player, but still it could be a bit snappier. This also makes testing playlists etc. painful. Fortunately files in RECORD directory, for example, are not indexed, so you can put there everything that doesn't need to be indexed.
Playlists are also read very slowly. If a playlist is long, reading it will take forever. The best way to cope with these problems, I think, is to use only small playlists. Better way to categorise files is to set their genres to your liking with Rhytmbox/Easytag. Genres are indexed during database update and can be accessed almost instantly.
One other observation, that I can't be sure about at this point, is that the player seems a bit faster when mp3's are scattered in different directories in root folder, rather than all in MUSIC-folder. It also seems that very long file names may cause some sporadic file system problems from time to time. So, I have decided to remap my music collection to the root folder of the player, in a few directories with short names, with no folders for albums and such. I have automated this with a script that rsyncs my music collection to the device.
I've also had this problem with file transfers from time to time, where nothing is copied to the player and the system says the file system is read only. I'm not sure but perhaps this has something to do with filenames starting with a dot. After that remounting and deleting the file sometimes helps:
A Few Further Notes
My firmware version was V01.01.11F, which at the time of writing this is the newest. So there was no need for firmware update. Apparently you can update firmware, however, without Windows just by dragging firmware .bin file to the device root in MSC mode. I haven't tested this.
I have some mp3 files with broken unicode characters in them. During this exercise I found that it is just easier to remove them from filenames than to get them working with Fuze and playlists. Here's one pretty demented way of doing just that:
That took some experiment to get working. If you have just one or two broken filenames, it is easier just to rename them in Nautilus. Unfortunately, I had tens of them. This command is probably not the ideal way to rename files, but again, it was good enough for me.
Some discussion on Sandisk forums:
Conclusions
At this point it is hard for me to say whether I would recommend this player for Linux owners. SanDisks attitude towards Linux seems somewhat positive and they deserve to be rewarded for that. On the other hand, their media conversion software is still Windows only and their video format is just silly. The player probably takes some time to get used to, but at this point it does seem slow. Also, managing a large number of media files is not very easy.
This page chronicles the several problems I have had with this player, many of them in no way related to Linux. In general, I'm not happy with the quality of its software, especially. But from my own experience and from what I gather from the forums, other players are not much better. My suggestion is to research your own purchases thoroughly beforehand.
I've recently purchased a SanDisk Sansa Fuze MP3 player. My old player has served me well for a very long time, but it's age is showing and I thought it was finally time to retire it. These are my notes on getting the Sansa Fuze working with my Acer laptop and Ubuntu Linux.
I selected SanDisk player after googling revealed encouraging reports about it's Linux compatibility and, especially, after I discovered that SanDisk explicitly mentions Linux in it's product specs. It also had some very nice features when compared to some of it's competitors, FM radio and voice recording for example. Also, I had seen some rumours about upcoming Ogg Vorbis support. Now, rumours are rumours and I believe when I see it, but ogg support would be absolutely fantastic.
Connecting
MP3 players in general use two different protocols to communicate with the computer. The older MSC protocol tends to work well with different operating system as the device shows up as a mass storage drive for the computer. Microsoft's newer MTP protocol, designed to implement DRM restrictions to players, usually requires special software. Then, of course, there are some variations of the two. For example, Microsoft apparently uses protocol called MTPZ for Zunes for some reason. Go figure.
Anyway, Sansa Fuze claims to support both of the aforementioned protocols, but only MSC for OSX and Linux. So I wanted to try that protocol first. I connected my player to my Acer 2970Z's USB port and waited. Nothing happened on the computer. The Sansa Fuze claimed it was "connected", but lsusb didn't show anything new. Oh well, I expected this much. So I tried this tip for another SanDisk player (removing kernel USB 2.0 driver with "sudo rmmod ehci-hcd", see also this post) and sure enough the player mounted immediately. At least now I knew my player could be mounted, even if at suboptimal speed. Fortunately I found out that the speed could be improved easily by reloading the kernel module after Fuze has been mounted ("sudo modprobe ehci-hcd"). This remounts the player and copying is much faster. Later I also discovered that if I use a passive USB hub the player seems to mount with no mod-magic from time to time, although not consistently.
Here's the command I use to get the player recognised. The sleep command gives the player some time to mount and increases the chances of this working slightly, I find:
sudo rmmod ehci-hcd; sleep 1s; sudo modprobe ehci-hcd
(Update: apparently, it at least sometimes suffices to use "lsusb".)
I copied a few mp3's and an ogg file to the root folder of the player. Mp3's worked, the ogg didn't (as expected, of course). I then copied much of my music library to the device. After doing this I discovered that m3u playlists showed up on the player, but they didn't work. More about this below.
MTP
Unfortunately, MSC mode does not show the content that had been preloaded to the player. So next, I tested the MTP support with MTPFS, which is supposed to enable drag'n'drop copy to MTP players from Linux. Apparently, it also has playlist support. So I disconnected the player, typed sudo apt-get install mtpfs, and set the player to MTP mode. I also reloaded the USB 2 kernel module just to see if it matters. Then I reconnected player. Nothing happened.
Mtpfs actually uses libmtp to connect to the player, so next I installed libmtp's mtp-tools with "sudo apt-get install mtp-tools" and ran mtp-detect, which promptly informed me that "No Devices have been found".
I proceeded to install the latest libmtp and mtp-tools from source (version 0.3.0). I found a helpful post at the Sandisk forums and followed it as far as it was applicable. The installation required some tweaking, as it apparently installed some files to non-standard locations and this left me for example with two versions of tool executables. Now I finally got mtp-detect to see my device. Amarok, Rhythmbox or mtpfs didn't see it, though. But I was still able to download the preloaded content from the player with mtp-tools.
Here is the one-liner I used to generate a command line for getting all the MTP mode tracks from the player:
(mtp-tracks | egrep -w '(Track ID|Origfilename)' |sed 's/[ \t]*$//'| sed 's/^.*D: /\" ; mtp-getfile /' | sed 's/^.*e: / \"/' | tr -d '\012' | colrm 1 4; echo \")
Not very elegant, very much ad hoc, but worked for me. As the command actually just prints a download command, which you can then check before pasting and running it, there should be no risk trying it. But, of course, I promise nothing and give no guarantees.
At this point I gave up on MTP for the moment and turned the player back to MSC mode. (Information from mtp-detect should probably be posted at libmtp bug tracker. I haven't got around doing this yet, unfortunately, as MTP isn't a must for this device. Perhaps someone else will find the time.)
Video
Getting video playback is a whole different story. Others have tried with no real success (see e.g. reqeya's post). First I copied the video file preloaded to the player and transferred it back with a new name. It worked. Then I tried copying some random avi files to the player. The player detected these files as video, but complained that they were in a wrong format. No surprise there.
Next I downloaded Windows software Sansa Media Converter from Sansa web site and tried it under Wine and in a virtual machine. Neither worked.
I tried numerous different things to get video working. Again and again, the player said that the media format is unsupported. I'm not very optimistic at the moment, as further investigation revealed that the player requires very specific format for the avi container. See the post I wrote about this for further details on the problem and the format used by the player.
I don't know much about video encoding and would appreciate if anyone else could give some advice on getting video work. For reference you might want to see further information on the video clip and this discussion. It's also worth noting that in the past others have had some success with earlier Sansa models and video, but the player's video format has been ridiculously specific. And they have of course changed the format for this model.
Let me know if you find anything useful.
Playlists
I already mentioned that m3u playlists seemed to be supported in principle. Some additional work is required, however, to get them fully functional. I tried creating a simple m3u file in the root folder of the device with only two entries. I created two versions of the file: normal one with a text editor and a dos version with unix2dos. Both showed up on the device but neither worked.
After some additional frustration I installed Easytag from Synaptic. This is how I got my first working playlist with a single song. I then copied this playlist to a different name, opened it in nano, changed the single entry in the file into a different file and saved. This also worked. Then I created my own list with identical input with nano and ran the file through unix2dos. After several tries I got this working too. As far as I know it is essential that the playlist has CR's in the end of every line:
$ cat -A list.m3u
#EXTM3U^M$
#EXTINF:259,song1.mp3^M$
song1.mp3^M$
The playlist must also use relative paths and Windows-style backslashes as directory separators.
Quite bizarrely, some of the stuff I tried seemed to work on and off. It seems that there were some random errors, possibly due to me yanking the usb cable on and off constantly. So, to be safe, you probably should wait for a while before detaching the cable, or you should unmount the device cleanly.
I have written some custom scripts that generate a few playlists automatically from my music collection. Unfortunately I noticed that Fuze read these very slowly and worked with them only after pushing menu button and back after the list had been loaded. Strange.
Slooow
The most notable annoyance with the player has been it's slow functioning. Apparently several gigabytes of music on a VFAT drive is just too much for it.
Especially slow is the "database update" after file transfer. This is mostly due to the numerous media files on the player, but still it could be a bit snappier. This also makes testing playlists etc. painful. Fortunately files in RECORD directory, for example, are not indexed, so you can put there everything that doesn't need to be indexed.
Playlists are also read very slowly. If a playlist is long, reading it will take forever. The best way to cope with these problems, I think, is to use only small playlists. Better way to categorise files is to set their genres to your liking with Rhytmbox/Easytag. Genres are indexed during database update and can be accessed almost instantly.
One other observation, that I can't be sure about at this point, is that the player seems a bit faster when mp3's are scattered in different directories in root folder, rather than all in MUSIC-folder. It also seems that very long file names may cause some sporadic file system problems from time to time. So, I have decided to remap my music collection to the root folder of the player, in a few directories with short names, with no folders for albums and such. I have automated this with a script that rsyncs my music collection to the device.
I've also had this problem with file transfers from time to time, where nothing is copied to the player and the system says the file system is read only. I'm not sure but perhaps this has something to do with filenames starting with a dot. After that remounting and deleting the file sometimes helps:
sudo mount -o remount,rw /media/SANSA\ FUZE/
A Few Further Notes
My firmware version was V01.01.11F, which at the time of writing this is the newest. So there was no need for firmware update. Apparently you can update firmware, however, without Windows just by dragging firmware .bin file to the device root in MSC mode. I haven't tested this.
I have some mp3 files with broken unicode characters in them. During this exercise I found that it is just easier to remove them from filenames than to get them working with Fuze and playlists. Here's one pretty demented way of doing just that:
ls --color=none -b1i | while read i; do find . -inum ${i%% *} -exec mv {} ${i#* }" \;; done
That took some experiment to get working. If you have just one or two broken filenames, it is easier just to rename them in Nautilus. Unfortunately, I had tens of them. This command is probably not the ideal way to rename files, but again, it was good enough for me.
Some discussion on Sandisk forums:
Conclusions
At this point it is hard for me to say whether I would recommend this player for Linux owners. SanDisks attitude towards Linux seems somewhat positive and they deserve to be rewarded for that. On the other hand, their media conversion software is still Windows only and their video format is just silly. The player probably takes some time to get used to, but at this point it does seem slow. Also, managing a large number of media files is not very easy.
This page chronicles the several problems I have had with this player, many of them in no way related to Linux. In general, I'm not happy with the quality of its software, especially. But from my own experience and from what I gather from the forums, other players are not much better. My suggestion is to research your own purchases thoroughly beforehand.
Subscribe to:
Posts (Atom)