Untitled Document

/usr/syno/etc.defaults/mimetypes.txt
Replace
avi video/avi
with
avi video/x-divx

Code: Select all
Replace
mp4 video/mp4
with
mp4 video/MP4V-ES

Replace
avi video/avi
with
avi video/mpeg

Whats missing are AVCHD, coverart, .divx container support.
* AVCHD videos (.M2TS) can be made to work by renaming to .mpg
* .DIVX videos can be made to work by renaming to .avi (Edit: Now with latest Synology firmware .divx works with PS3)
* M4V videos will work if you rename to .MP4 though the video must be H.264 and audio AAC, anything else Apple software sticks into M4V wont work.

» Optimize your Synology NAS for downloading

I recently bought a NAS so my data is safe & available, with the benefit of being low power / noise / heat. I've considered Netgear, QNAP, but decided to go for a Synology as it was affordable, still had a big community, decent reviews & Time Machine support.

I wanted 4 bays so that I could use RAID5 and only lose 25% space on fault tolerance instead of (RAID1) 50%. Synology has 2 offerings in the 4-bay home-user range this year: the DS411+ (fast) and the DS411j (slow).

I figured as long as it can blast a bandwidth adequate for 1080p over my network, I'd save myself some money (300$ vs 600$), heat and power consumption that come with the more powerful + version.

However now that it's here I want it do download from newsgroups and am running into performance issues with my junior edition.

No worries. With a little bit of hacking you can squeeze just enough performance out of this thing to make sense of it all.

Here's how I turned my budget NAS that's mediocre at 8 things into a more powerful one that's good at 3 things: downloading / file serving / backups.

Warning

This article assumes you're somewhat skilled in Linux. By applying these suggestions you could seriously mess up your Disk Station.

I'm doing this on a DS411j running DSM 3.0. Your mileage may vary.

Downloading

In an earlier article I described how to install SABnzbd. After testdriving it for a while I was never able to get it to download above 3MB/s (2 average). Where as nzbget (the program used by Synology's own Download Station), peaks at 8MB/s (6 average).

Although I really like that SABnzbd automatically unpacks your downloads, these speed differences made me decide to go back to nzbget. The j is just not powerful enough to do SABnzbd at these speeds, and I can write auto-unpackers myself.

Optimal Config

I found that optimal speeds can be reached by letting your Synology download with 8 connections on 1 single download. With these settings the load reaches 11, so don't expect your NAS to do anything else while it's busy. But at least you're saturating your connection.

If you want it to multitask, limit it to 1 connection on 1 single download at any time, but you won't see it peak beyond 2MB/s.

If you use it for torrents as well, you don't want 1 slow torrent blocking the rest of the queue. In that case, set it to 2 to 3 connections with 2 to 4 threads each for optimal downloading.

Turn off unused protocols

Decide on 1 file-sharing protocol (I chose Mac File service cause all my systems speak it and use Time Machine, but SMB/Windows is typically the right choice). Disable the rest in your configuration panel, saving a few precious MBs of RAM.

This is all just done from your web-interface.

SSH access

Before you can do any hacking on your Synology, turn on SSH access in the web-interface's control panel. You can now type: ssh root@166.109.54.46. Followed by sh. The root password is the same as admin password.

AppStore :)

Get your hands on ipkg, which is like your Synology's secret AppStore. From here on, it's much easier to install cool additional software.

Turn off media indexers to free up CPU & Memory

When I logged in to see what was eating up my NAS' resources, I saw a lot of processes running that I don't need such as thumbnail generators and media indexers (ffmpeg & convert).
They were endlessly consuming 100% CPU, leaving nothing for my other tasks.

Any currently available NAS is a terrible media streamer. And that's ok, just get yourself an AC Ryan ($80) or Boxee Box ($250) to do that instead and dedicate your NAS to less tasks.

In my case that meant killing off all these wannabe media processes that are eating up your poor handheld CPU with 128MB RAM (every MB we'll save from this point forward counts to faster download speeds :)

So if you don't use the Photo/Media/iTunes station and would like more power for other tasks, consider turning off indexers:

1) Turn off all services in the bottom configuration panel (iTunes, everything except Download Station, unless you're going to use SABnzbd for this)

2) Login as root via SSH and stop all indexing:
/usr/syno/etc/rc.d/S??synoindexd.sh stop
/usr/syno/etc/rc.d/S??synomkflvd.sh stop
/usr/syno/etc/rc.d/S??synomkthumbd.sh stop
killall -9 convert
killall -9 ffmpeg
# If you don't use Download Station (but e.g. SABnzbd instead):
# /usr/syno/etc/rc.d/S??pgsql.sh stop

vi /etc/synoinfo.conf
advrexpire_mediaservice = 150

3) Make sure they won't restart on your next reboot.
chmod -x /usr/syno/etc/rc.d/S??synoindexd.sh
chmod -x /usr/syno/etc/rc.d/S??synomkflvd.sh
chmod -x /usr/syno/etc/rc.d/S??synomkthumbd.sh
# If you don't use Download Station (but e.g. SABnzbd instead):
# chmod -x /usr/syno/etc/rc.d/S??pgsql.sh

Hint) After a DSM firmware upgrade, you need to repeat these steps.

Custom cleanup & rename script cause SAB is too slow

Building your own cleanup scripts can be fun (and risky). If you want to get into it, you'll need some system tools at your disposal.

Here's what I cooked up to take care of my downloads. It runs every 15 minutes by cron, will remove broken downloads, unpack complete downloads, move lonely files 1 directory up, delete a bunch of unwanted extensions, etc. It makes a few assumptions (e.g. downloads must be in /volume1/downloads), so be sure to only use it for inspiration.

It's a work in progress, and improvements are more than welcome.

Downloadstation CLI

To have your Synology scan a directory for new download tasks, you can use Downloadstation CLI.
ipkg install python24 py-pgsql py24-mx-base
wget -O- http://downloadstation.jroene.de/downloadstation > /opt/bin/downloadstation
chmod 755 /opt/bin/downloadstation

With the command
downloadstation add <nzbfile>

The download will be added to the queue. If you use an adaptation of my unpacker script, it will already automatically scan /volume1/downloads/_queue for any new torrent or nzb task.

Tools

These programs may take up a little bit of space, but won't be active in memory until you call upon them (except for cron), so feel free to install without performance loss:
ipkg install vim bash bash-completion less rsync mtr \
sudo tshark htop openssl mlocate perl ack hdparm sysstat dstat \
bzip2 unrar unzip zlib p7zip

curl -k https://github.com/timkay/solo/raw/master/solo -o /usr/bin/solo
chmod a+x /usr/bin/solo

Optionally do ipkg install clamav so you can run clamscan on freshly downloaded files and check them for viruses (I decided not to).

Renaming files

There's a neat program called tvnamer that will rename all your TV series files.

Install:
ipkg install python25 py25-setuptools git
cd /volume1/@tmp
git clone https://github.com/dbr/tvnamer.git
cd tvnamer
python setup.py install
ln -s /opt/local/bin/tvnamer /usr/bin/tvnamer

Use:
tvrenamer -r /volume1/video/tv

FileBot is even better but requires a GUI.

Crontab

Crontab works slightly different than on more high-level Operating Systems.

Here's how to edit your crontab:
vim /etc/crontab

Every job needs a user prefix. e.g. root:
*/15 * * * * root /usr/bin/solo -port=1111 /volume1/video/unpacker.sh 1>&2 > /volume1/@tmp/unpacker.log

When you're done editing the new crontab, reload it by executing:
/usr/syno/etc.defaults/rc.d/S??crond.sh stop
/usr/syno/etc.defaults/rc.d/S??crond.sh start

Tmux or Screen

If you start programs from within tmux, you can close your SSH session without killing it. You can check back later on it with tmux attach || tmux.

This makes it perfect to run cleanup/rename scripts in while you're still experimenting and need to check up on them regularly.

Tmux similar to screen, but I think it's a bit easier to deal with (just tmux attach || tmux is all).

However screen is a lot easier to install thanks to ipkg, so pick your poison.

Screen
ipkg install screen

Tmux
ipkg install libevent optware-devel ncurses-dev

# http://forum.synology.com/enu/viewtopic.php?f=90&t=30132
mkdir /opt/arm-none-linux-gnueabi/lib_disabled
mv /opt/arm-none-linux-gnueabi/lib/libpthread* /opt/arm-none-linux-gnueabi/lib_disabled
cp /lib/libpthread.so.0 /opt/arm-none-linux-gnueabi/lib/
cd /opt/arm-none-linux-gnueabi/lib/
ln -s libpthread.so.0 libpthread.so
ln -s libpthread.so.0 libpthread-2.5.so

cd /volume1/@tmp
wget http://sunet.dl.sourceforge.net/project/tmux/tmux/tmux-1.4/tmux-1.4.tar.gz
tar -zxvf tmux-1.4.tar.gz
cd tmux-1.4
export CC=gcc
export CFLAGS="-L /opt/lib -I /opt/include/ncurses"
./configure --prefix=/opt # prefix is not supported. So we'll need some symlinks
make # This will take a while
make install
ln -s /opt/lib/libevent-1.4.so.2 /usr/lib/libevent-1.4.so.2
ln -s /opt/share/terminfo/* /usr/share/terminfo/

----------------------------------------------------------------------

DISCLAIMER: This walkthrough is solely for my reference. If you fuck-up anything doing it, you are on your own ! Love, peace and best wishes.

Back to revival
Now first what is running on the syno can be seen by running the ps command. The output is like this:

Poseidon> ps
PID Uid VmSize Stat Command
1 root 16 S init
2 root SW [keventd]
3 root SWN [ksoftirqd_CPU0]
4 root SW [kswapd]
5 root SW [bdflush]
6 root SW [kupdated]
7 root SW [kinoded]
8 root SW [mtdblockd]
121 root SW [kjournald]
196 root SW [khubd]
349 root 148 S /sbin/syslogd -S
351 root 4 S /sbin/klogd
380 root 28 S < /usr/syno/bin/findhostd
407 root SW [kjournald]
459 root 56 S /usr/syno/sbin/hotplugd
485 root 76 S /usr/sbin/inetd
538 admin 228 S /usr/syno/pgsql/bin/postgres -D /volume1/@database/pg
539 root 788 S /usr/syno/bin/scemd
582 nobody 188 S /opt/sbin/mt-daapd -c /opt/etc/mt-daapd/mt-daapd.conf
583 nobody 3800 S /opt/sbin/mt-daapd -c /opt/etc/mt-daapd/mt-daapd.conf
597 nobody 3800 S /opt/sbin/mt-daapd -c /opt/etc/mt-daapd/mt-daapd.conf
624 root 4 S /sbin/getty 115200 console
626 admin 208 S postgres: writer process
627 admin 72 S postgres: wal writer process
828 root 64 S N /usr/syno/sbin/synoindexd
835 root 64 S N /usr/syno/bin/synomkthumbd
895 nobody 3800 S /opt/sbin/mt-daapd -c /opt/etc/mt-daapd/mt-daapd.conf
896 root 656 S /usr/syno/sbin/nmbd -D
946 admin 24 S postgres: admin photo [local] idle
982 root 504 S /usr/syno/sbin/smbd -D
985 root 20 S /usr/syno/sbin/smbd -D
1050 root 84 S /usr/syno/sbin/atalkd
1062 root 80 S N /usr/syno/sbin/synomkflvd
1128 root 96 S /usr/syno/apache/bin/httpd -f /usr/syno/apache/conf/h
1188 root 328 S /usr/syno/apache/bin/httpd
1253 root 460 S /usr/syno/sbin/mDNSResponder -f /tmp/mDNSResponder.co
1256 root 460 S /usr/syno/sbin/mDNSResponder -f /tmp/mDNSResponder.co
1258 root 460 S /usr/syno/sbin/mDNSResponder -f /tmp/mDNSResponder.co
1264 root 336 S /usr/syno/sbin/ftpd -D
1269 root 68 S /usr/syno/apache/bin/httpd -f /usr/syno/apache/conf/h
1270 root 68 S /usr/syno/apache/bin/httpd -f /usr/syno/apache/conf/h
1271 nobody 308 S /usr/syno/apache/bin/httpd
1272 nobody 312 S /usr/syno/apache/bin/httpd
1273 nobody 312 S /usr/syno/apache/bin/httpd
1280 root 44 S /usr/syno/sbin/afpd -c 256 -g guest -n Poseidon:AFPSe
3257 nobody 316 S /usr/syno/apache/bin/httpd
3259 root 68 S /usr/syno/apache/bin/httpd -f /usr/syno/apache/conf/h
3260 root 68 S /usr/syno/apache/bin/httpd -f /usr/syno/apache/conf/h
3261 root 68 S /usr/syno/apache/bin/httpd -f /usr/syno/apache/conf/h
3668 root 232 S crond -S -b
3756 nobody 320 S /usr/syno/apache/bin/httpd
4618 root 1988 S sshd: root@ttyp0
4622 root 376 S -ash
4630 root 764 R ps

syno scripts
The syno start/stop scripts are located in the directory "/usr/syno/etc/rc.d". When dissecting these scripts you can see that there is a central configuration file called /etc.defaults/synoinfo.conf which stores the supported features and the configuration file which enables/disables these features at /etc/synoinfo.conf aka "config file". Easier it is to disable the rc scripts if you want to get rid of these features. Create a directory /usr/syno/etc.defaults/rc.off and move all the unwanted S[0-9]{2}[A-Za-z-]*.sh scripts there.

mDNSResponder
Something that is realted with bonjour/apple: "The open source mDNSResponder project contains the Bonjour service discovery implemenation that's built into Mac OS X and Bonjour for Windows." Since no apples here, there is no use for it.
The process: mDNSResponder
The script: S99mDNS.sh
The parameter: runitunes, runafp, runaudiostation (If any of these is set to yes in config it will start)
The stop: /usr/syno/etc/rc.d/S99mDNS.sh stop
The safest way of getting rid of this daemon it is to set all these parameters to "no" in the config file.

atalkd
Something for the Mac's Since no apples here, there is no use for it.
The process: afpd atalkd papd
The script: S81atalk.sh
The parameter: runafp
The stop: /usr/syno/etc/rc.d/S81atalk.sh stop
Again to be on the safe side, set runafp to "no" in the config file.

apache
All the apache web servers. For the user. For the admin. I'll be using nginx with php. And configuring this beast through ssh and config files is enough for me.
The process: httpd
The script: S97apache-sys.sh, S97apache-user.sh
The parameter: runweb
The stop: /usr/syno/etc/rc.d/S97apache-user.sh stop; /usr/syno/etc/rc.d/S97apache-sys.sh stop
Seems like the system web server must be stopped manually. Might get handy when it's up after restart.

synomkthumbd
Thumbnails. You can't even configure if it will run. In the script it simply checks if the binary exists and when yes, then run's it.
The process: synomkthumbd
The script: S77synomkthumbd.sh
The parameter: NONE
The stop: /usr/syno/etc/rc.d/S77synomkthumbd.sh stop

synoindexd
Indexing. Something. Whatever. If you don't know what it is you probably don't need it :-)
The process: synoindexd
The script: S66synoindexd.sh
The parameter: supportmediaservice, supportphoto, supportitunes, support_audio in /etc.defaults/synoinfo.conf (If any of these is set to yes in config it will start)
The stop: /usr/syno/etc/rc.d/S66synoindexd.sh stop

synomkflvd
Videostation. flv is a container format for adobe flash player.
The process: synomkflvd
The script: S88synomkflvd.sh
The parameter: supportvideostation in /etc.defaults/synoinfo.conf
The stop: /usr/syno/etc/rc.d/S88synomkflvd.sh stop

postgres
SQL database. If this command '/usr/syno/bin/servicetool —get-service-path pgsql' returns a path '/volume1/@database' (e.g. something that is not equal to zero' the database will start.
The process: postgres
The script: S20pgsql.sh
The parameter: NONE
The stop: /usr/syno/etc/rc.d/S20pgsql.sh stop

findhostd
Find something. Probably other syno boxes ? You can't even configure if it will run. In the script it simply checks if the binary exists and when yes, then run's it.
The process: findhostd
The script: S98findhostd.sh
The parameter: NONE
The stop: /usr/syno/etc/rc.d/S98findhostd.sh stop

ftpd
file transfer protocol. Nice thing. You might keep it. I share everything through samba in my home networks. And there is always sftp anyway (think winscp)
The process: ftpd
The script: S99ftpd.sh
The parameter: runftpd
The stop: /usr/syno/etc/rc.d/S99ftpd.sh stop

Ok, we are now down to this list of processes:

Poseidon> ps
PID Uid VmSize Stat Command
1 root 44 S init
2 root SW [keventd]
3 root SWN [ksoftirqd_CPU0]
4 root SW [kswapd]
5 root SW [bdflush]
6 root SW [kupdated]
7 root SW [kinoded]
8 root SW [mtdblockd]
121 root SW [kjournald]
196 root SW [khubd]
349 root 172 S /sbin/syslogd -S
351 root 4 S /sbin/klogd
407 root SW [kjournald]
459 root 56 S /usr/syno/sbin/hotplugd
485 root 76 S /usr/sbin/inetd
539 root 788 S /usr/syno/bin/scemd
582 nobody 188 S /opt/sbin/mt-daapd -c /opt/etc/mt-daapd/mt-daapd.conf
583 nobody 3800 S /opt/sbin/mt-daapd -c /opt/etc/mt-daapd/mt-daapd.conf
597 nobody 3800 S /opt/sbin/mt-daapd -c /opt/etc/mt-daapd/mt-daapd.conf
624 root 4 S /sbin/getty 115200 console
895 nobody 3800 S /opt/sbin/mt-daapd -c /opt/etc/mt-daapd/mt-daapd.conf
896 root 660 S /usr/syno/sbin/nmbd -D
982 root 504 S /usr/syno/sbin/smbd -D
985 root 20 S /usr/syno/sbin/smbd -D
3668 root 232 S crond -S -b
4618 root 2028 S sshd: root@ttyp0
4622 root 424 S -ash
4870 root 764 R ps

samba will probably get replaced by a optware samba in the future (nmbd, smbd).
maybe scemd will be removed ?
xinetd instead of inetd ? Anyway inetd won't run that much of processes so disabling it all-together might be an idea (and run all the other daemons instead).

------------------------------------------------------------------------------------------

Overview

¦
Synology Website: www.synology.com

¦
Current Firmware for DS-106j: DSM 2.1-0839; Build Date: 2009/03/25

Tweaking iTunes Music Indexing

¦
Activate iTunes Service

¦
Login to the DiskStation using telnet or ssh

¦
Remove music folder: rm /volume1/music
¦
Symlink the folder to your favorite location: ln -s /volume1/public/my_music /volume1/music
¦
Re-run the indexer to manually update the index: synoindex -A /volume1/music
Make sure not to put a slash (/) at the end of the path! synoindex -A /folder is not the same as synoindex -A /folder/. Read below for more information on synoindex.

The index is only kept up to date as long as you use Synology’s “official” interfaces (ftp, smb, etc.) for adding/removing media files. Whenever you add/remove files manually using the command line you have to fix the index using synoindex.

Synology Indexing

The meta information of media files (music, photos, videos etc.) is kept in a database that is updated automatically by synoindexd. The database is PostgreSQL and can be found at /volume1/@database. The indexing daemon can be manipulated from the command line by synoindex.

Index database

To access the database use the following command:
DiskStation> /usr/syno/pgsql/bin/psql mediaserver admin
Welcome to psql 8.2.5, the PostgreSQL interactive terminal.

Type: \copyright for distribution terms
\h for help with SQL commands
\? for help with psql commands
\g or terminate with semicolon to execute query
\q to quit

mediaserver=#

usage:
Add: synoindex -a filename
Delete: synoindex -d filename
Add folder: synoindex -A folder
Delete folder: synoindex -D folder
Rename/move file/folder: synoindex -N newfullpath oldfullpath
Update Photo Images: synoindex -U photo
Get from DB: synoindex -g filename -t [video|music|photo|playlist]

Additionally there is another undocumented option -R:
synoindex -R [video|music|photo|playlist|all]

It seems to re-index the given table but I am not sure what it does exactly. If you removed or renamed a folder from the console and want to fix the index the -R option doesn’t help you. In that case you have to delete the old folder using -D and then add the new folder using -A.

When deleting a folder make sure not to add a trailing slash (/) to the path! To remove the index from all files in the music share type:
synoindex -D /volume1/music

Orphans in the database

Whenever you rename or move files manually using the console the database still contains information to these files which is no longer valid. This is especially annoying when you rename files or folders and then run synoindex -A /your_folder to update the database. The information for the renamed files will then be added a second time to the database. In iTunes the files will show up twice but only one link (the new one) will still work.

Unfortunately synoindex -R does not remove entries for files from the database that do no longer exist. The only way to fix this is to use synoindex -D on the old folder and then re-index using synoindex -A.

The following script queries all files from the music database and verifies if the file still physically exists. If not a message is printed to STDOUT. If started with -f the script also removes these files from the index database.
Usage: ./remove_orphans.sh [-f]
#!/bin/sh
# Usage: ./remove_orphans.sh [-f]

[ "$1" = "-f" ] && REMOVE=1

IFS='
'

for t in `/usr/syno/pgsql/bin/psql mediaserver admin -tA -c "select path from music;"`; do
if [ ! -f "$t" ]; then
echo "MISSING: $t"
[ -n "$REMOVE" ] && synoindex -d "$t"
fi
done