2020 NAS – Plex, nomachine, Crashplan

Posted on by rcragun

After about a year and a half with my previous NAS (see here), I decided it was time for an upgrade. The previous NAS had served dutifully, but it was no match for 4K video (I don’t have a lot of it), it took forever to transcode files when I wanted to synchronize them with a device using Plex, and it couldn’t transcode pretty much any videos in real-time. For playing up to four files simultaneously that didn’t need to be transcoded, it worked like a champ. But it just couldn’t handle all the scenarios I was throwing at it. So, it was time for an upgrade.

Hardware

The major change, of course, was the hardware. Here are the specs for the new NAS:

Case: Fractal Design Define R5 – Mid Tower ATX case
RAM: Corsair LPX 32GB (2x16GB) 3200MHz C16 DDR4 DRAM
GPU: EVGA GeForce GTX 1060 3GB
Motherboard: ASUS AM4 TUF Gaming X570
CPU: AMD Ryzen 5 3600 6-Core
Powersupply: Fractal Design Ion+ Platinum 860W PSU
SSD: Samsung 850 EVO 250GB
Hard Drives: 4 WD Blue 4TB 5400 RPM Hard Drives

Since I was repurposing the SSD and hard drives from my previous NAS, I didn’t buy those for this build. But here is the estimated cost of the NAS assuming those were included: $1,357.47 + tax.

(NOTE: Full disclosure, the links above are affiliate links to Amazon.)

A couple of notes on my hardware choices…

The case is amazing. It’s big, roomy, and extremely well-designed. It is also absolutely silent and does a great job with cable management. It also has plenty of room for hard drives, which I absolutely love. It’s also understated. I don’t need flashing lights with my NAS. I need a discrete box that makes no noise.

My new fileserver. It’s absolutely silent. It may be a little large, but that gives me plenty of space for the components I want.

I had originally considered a different GPU, the PNY Quadro P2000. A lot of Plex forums suggested it was a real beast when it came to hardware transcoding for video. By the time I put this together, these were really hard to find and very expensive. I looked around and the GeForce GTX 1060 had higher benchmarks at a much lower price. You can see my testing data below to see how well it works and whether I made the right choice.

I also considered an AMD Threadripper over a standard Ryzen, but I also realized that I was passing the video transcoding to a GPU with this build (my last one used the CPU), which meant I really didn’t need that much CPU power. Again, you can see in the benchmarks below how well this paid off.

The remaining hardware choices were pretty straightforward. The SSD is small because it only houses the OS. All the storage is on the hard drives. I kept the ZFS raid (see details below) and can upgrade it by simply buying bigger hard drives in the future and swapping the smaller ones out, making this NAS fairly future proof for at least the next few years.

Software

I changed very little from the previous NAS when it comes to software. I installed the latest Kubuntu LTS, 20.04. I am running the latest versions of Plex, Crashplan, tinyMediaManager, kTorrent, and nomachine. I am using NFS to access my server directly from other computers within the network. These are all detailed in my previous build.

I also kept my ZFS file system. Conveniently, I was able to transfer the raid right over to my new NAS, which I detail below.

Test Results

To test how well my new build did, I ran a couple of tests. First, I tried to simultaneously stream a 4K video to 5 devices – my TV, two Google Pixel 3 phones, a Google Pixel phone, and a laptop (all on my home network). The Google Pixel couldn’t handle the 4K video, so I ended up streaming a different 1080p video instead. My previous NAS couldn’t stream a single 4K video. This one handled all 5 streams like a champ, as shown in the video below:

I also wanted to make sure that my Plex server was using the GPU for converting videos, so I took that same 4K video and set it to synchronize with my phone at 720p. On my previous NAS, converting a 31gb 4K video to a 720p video would have taken hours. Plex passed the conversion on to the GPU and it took less than 10 minutes.

To make sure your Plex server is using your GPU, you need to set it to do so in the settings. It’s under Settings->Transcoder. Click on “Use hardware acceleration when available” and “Use hardware-accelerated video encoding.”

NOTE: To monitor the CPU, I used Ksysguard, which ships with Kubuntu. To monitor the GPU, I used “nvtop,” which can be installed from most distribution repositories.

Headless Monitor Issue

One of the problems I have been struggling with for months now (and had a similar issue with my previous NAS) was setting up a virtual screen/monitor. Here’s the problem…

I actually do the initial installation of the operating system using a monitor that I plug into the NAS. But I don’t want to keep a monitor out, sitting near my NAS/fileserver all the time. I want the NAS to be “headless” – meaning no monitor is attached. Once I get the operating system installed, I first install SSH (just in case I screw something up, ’cause you’re always going to screw something up). Then I install my remote management software, nomachine. Once I have nomachine installed, I can unplug the monitor, mouse, and keyboard and take care of everything else remotely. However, I found out with my previous NAS build, that the X server that manages sessions on Kubuntu, doesn’t set up a monitor if it doesn’t detect one being attached physically. In other words, if I restart the computer with no monitor attached, the X server correctly detects that there is no monitor attached and doesn’t set up a monitor in the X session.

As a result, when I would try to remotely connect to my NAS using a VNC client, the client would have to try to generate a monitor and the result was always problematic – either nothing would show up or it would be really slow and sluggish. I get that the X server doesn’t want to use resources to run a monitor if there isn’t a monitor connected. That’s a feature, not a bug. But it is a challenge to solve this problem. With my new NAS, it took me about 2 hours to finally figure out a solution. So I don’t have to go through this again, I’m going to post the solution here in detail.

With my previous NAS, I didn’t have a graphics card installed. I let the CPU do all the processing, which meant I needed a different solution than the one I had employed before (here’s the old solution). With my new NAS, I wanted a graphics card so I could pass off some of the video processing to the GPU instead. As noted above, I bought an NVIDIA graphics card, which Kubuntu recognized when I installed the OS and appropriately added the recommended NVIDIA drivers. My graphics card also runs the monitor. This was the key to solving my headless monitor issue.

Basically, what you need to do is tell the X Server to create a monitor even though one isn’t attached. You do this by creating a “xorg.conf” file that is stored in the /etc/X11/ directory. Once the NAS is up and running, you can actually let the NVIDIA X Server Settings application create the necessary xorg.conf file, for the most part. Here’s what you do. With the NVIDIA drivers installed and the NVIDIA X Server Settings application installed, launch the NVIDIA X Server Settings application. Go to the second option down on that window that reads, “X Server Display Configuration.”

Near the bottom of that window is a button that reads, “Save to X Configuration File.” This is nearly the answer to all our problems.

When you click on that, it will pop up a window that will allow you to preview the code it is going to add to /etc/X11/xorg.conf, which is the X server’s configuration file for the monitor. That code will be based on your current monitor – whatever is currently physically attached to the computer. Go ahead and follow the prompt and it will generate the xorg.conf file.

However, you now need to make two modifications to the file. I use “nano” to edit text files:

sudo nano /etc/X11/xorg.conf

First, you need to add the following to the Device Section of the xorg.conf file:

Option “AllowEmptyInitialConfiguration”

This code tells the X Server that it’s okay to create a screen/monitor without a physical monitor connected. (Got this information from here).

Second, you need to add the dimensions of the virtual screen you want to set up to the “Display” SubSection of the “Screen” Section of the xorg.conf file, like this:

Virtual 1440 900

This creates a virtual monitor/screen with the dimensions indicated. (Got this information from here). Here is the complete xorg.conf file I am using for my NAS:

# nvidia-settings: X configuration file generated by nvidia-settings
# nvidia-settings:  version 440.64

Section "ServerLayout"
    Identifier     "Layout0"
    Screen      0  "Screen0" 0 0
    InputDevice    "Keyboard0" "CoreKeyboard"
    InputDevice    "Mouse0" "CorePointer"
    Option         "Xinerama" "0"
EndSection

Section "Files"
EndSection

Section "Module"
    Load           "dbe"
    Load           "extmod"
    Load           "type1"
    Load           "freetype"
    Load           "glx"
EndSection

Section "InputDevice"
    # generated from default
    Identifier     "Mouse0"
    Driver         "mouse"
    Option         "Protocol" "auto"
    Option         "Device" "/dev/psaux"
    Option         "Emulate3Buttons" "no"
    Option         "ZAxisMapping" "4 5"
EndSection

Section "InputDevice"
    # generated from default
    Identifier     "Keyboard0"
    Driver         "kbd"
EndSection

Section "Monitor"
    # HorizSync source: edid, VertRefresh source: edid
    Identifier     "Monitor0"
    VendorName     "Unknown"
    ModelName      "Acer AL1917W"
    HorizSync       31.0 - 84.0
    VertRefresh     56.0 - 76.0
    Option         "DPMS"
EndSection

Section "Device"
    Identifier     "Device0"
    Driver         "nvidia"
    VendorName     "NVIDIA Corporation"
    BoardName      "GeForce GTX 1060 3GB"
    Option	   "AllowEmptyInitialConfiguration"
EndSection

Section "Screen"
    Identifier     "Screen0"
    Device         "Device0"
    Monitor        "Monitor0"
    DefaultDepth    24
    Option         "Stereo" "0"
    Option         "nvidiaXineramaInfoOrder" "DFP-0"
    Option         "metamodes" "nvidia-auto-select +0+0"
    Option         "SLI" "Off"
    Option         "MultiGPU" "Off"
    Option         "BaseMosaic" "off"
    SubSection     "Display"
        Depth       24
	Virtual 1680 1050
    EndSubSection
EndSection

With the above xorg.conf file (which is located in /etc/X11/, so, /etc/X11/xorg.conf), I can then shutdown my NAS, unplug the mouse, keyboard, and monitor, then restart it and use nomachine to completely control the NAS. Here’s how it looks when I pull it up:

This approach will only work if you have an NVIDIA graphics card. If you don’t, you’ll need to take a different approach. This site may point you in the right direction.

UPDATE 2020-07-13: Back to XFCE

Well, I’m back to using XFCE as my desktop environment on my NAS/file server. An update completely wiped out KDE making it throw errors every time I tried to reboot the file server and not even showing the desktop. KDE is just not the right option for a NAS/file server. I had the same issue with my previous file server. Lesson learned. From now on, ultra lightweight XFCE will be my desktop environment of choice from my file server.

How to install XFCE. First, install tasksel:

sudo apt-get install tasksel

Then run tasksel and select Xubuntu desktop:

sudo tasksel

When prompted, select “lightdm” as your display manager of choice:

Go ahead and restart and you’ll be good to go. (NOTE: I’m still using the same xorg.conf file and it’s working.)

Also, I had to manually remove the NVIDIA drivers then reinstall them with the monitor connected.

ZFS Transfer

As I detailed in my previous NAS build, I opted to go with ZFS for storing my files using a raidz2 array (4 x 4tb drives). It provides, IMO, the optimal balance of speed and redundancy. With just over 4tb of files on my ZFS array in my old NAS, I didn’t really want to have to copy all of those files over to the new. As it turns out, ZFS has the ability to move physical disks to a new device and recreate a ZFS pool. Hallelujah!

On the old NAS, I shut down all connections to my NAS from external devices, restarted the machine, and then used the following command:

sudo zpool export

This basically sets up the ZFS pool so it is ready to be transferred to another machine. I then physically removed all four 4tb drives from the old NAS, moved them to the new one, and got everything ready to import the new NAS. I installed the following packages first:

sudo apt-get install zfsutils-linux nfs-kernel-server zfs-initramfs watchdog

The above packages also installed the following packages:

libnvpair1linux libuutil1linux libzfs2linux libzpool2linux zfs-zed keyutils libnfsidmap2 libtirpc-common libtirpc3 nfs-common rpcbind

With those installed, I checked to see if I had any pools (I didn’t, of course, I hadn’t set any up):

zpool status

Then, I used the following command to look for the pool that was spread across the four hard drives:

sudo zpool import

Beautifully, it immediately found the pool (ZFSNAS) and gave me a bunch of useful information:

pool: ZFSNAS
id: 11715369608980340525
state: ONLINE
status: The pool was last accessed by another system.
action: The pool can be imported using its name or numeric identifier and the '-f' flag.
see: http://zfsonlinux.org/msg/ZFS-8000-EY
config:
 ZFSNAS      ONLINE
      raidz2-0  ONLINE
        sdd     ONLINE
        sdb     ONLINE
        sde     ONLINE
        sdc     ONLINE

Honestly, I cheered when I saw this. It was going to work. I then used the following command to import my pool:

sudo zpool import -f ZFSNAS

Then, just to be sure everything worked, I checked my pool status:

zpool status

And up popped my zpool. It did tell me that I needed to upgrade my pool, which I proceeded to do. And, with that, my pool from my old NAS was transferred to my new NAS intact. No need to copy over the files. ZFS is definitely the way to go when it comes to NAS file storage. I’m increasingly happy with ZFS!

NOTE: I bought a 10gb external drive and copied everything onto it just to be extra cautious. Turns out, I didn’t need to do that. But, better safe than sorry.

Mounting Fileserver Across the Internet

This trick blew my mind. I am using EasyDNS to be able to VNC into my fileserver from wherever I am in the world (since I don’t have a static IP from my ISP). Nomachine works like a champ across the internet. But I had a crazy thought the other day, “Wouldn’t it be nice if I could actually mount my fileserver directly onto my work computer?” I don’t typically need to do this but I have been working with some files on my fileserver for a project and wanted direct access. Turns out, you can (thank you StackExchange). And so easily it will blow your mind. Linux rocks!

Since I have already set up SSH access on my fileserver, all I needed to do was learn about this as I already had all the packages installed. (If you haven’t set up SSH on your fileserver and opened the relevant ports on your home network, do that first). Then, on my computer at work, I created a new folder in my home directory (/home/ryan/fileserver/). I then used the following command at the terminal to mount my fileshares on my work desktop:

sshfs ryan@myfileserver.easydns.org:/directory/to/share /home/ryan/fileserver

You obviously need to change your username for your fileserver, which comes before the “@”, and need to have the IP address of your fileserver (what comes after the “@” sign). You also need to know what directory from your fileserver you want to mount (comes after the :/) and the folder where you want to mount it (the last piece). Once you have everything set up correctly, hit enter and it will ask you for your password. Once I entered my password, all of the files on my fileserver were securely shared over SSH to my work desktop:

Those are the main directories from my fileserver at home.

Note, this isn’t a permanent share. Once you shutdown your work computer, the share will go away. But it’s just a quick repeat of that command and you’ll have the files mounted right back to your computer. Also, the speed with which you can access the files will depend on your home and work networks. Luckily, mine are both very fast. So, it’s basically like being on my home network as I work with the files I need. Absolutely amazing!

 4,932 total views,  25 views today

Plex on Linux – Video Files Not Detected

Posted on by rcragun

I added a number of video files to my Plex server the other day and, when I checked Plex, some of them had not shown up in the corresponding library. I tried the obvious solutions. First, I selected the options for the library and then chose “Scan Library Files”:

That will often solve the problem, but it didn’t. Then I went for the bigger ask for my Plex server and selected “Manage Library -> Refresh all Metadata”:

That didn’t solve the problem either. Since my Plex server is on Linux, it took me a minute to think it through, but I wondered if the files I had put into the library had the appropriate permissions for Plex.

(Quick aside for those not used to Linux permissions. All files and folders on Linux have permissions assigned to them that dictate what can and cannot be done with them. Basically, it’s a 3×3 set of permissions. There are three options for what can be done with a file: Read, Write, Execute. Then there are three categories of those who can interface with the file: Owner, Group, Others. Thus, you can set the Owner to be able to Read, Write, and Execute a file while not allowing the Group or Others to do the same. Any combination of these permissions is theoretically possible.)

A quick check of the files on my Plex server showed that, indeed, some of the files had permissions settings that were likely interfering with Plex’s ability to index the files:

In the image above, you can see that the permissions for the two lower files (the first and second half of the Sevilla vs CFR Cluj game) are 600 with me as the owner and not “plex.”

My immediate response was to change the permissions to make them fully accessible to make sure that Plex could read them. To do that, I used the following command:

sudo chmod 777 -R /plexserver/videos/newfiles

Security gurus will freak out about this as I basically made those files accessible with no restrictions whatsoever. But, it did solve the problem with Plex. As soon as I changed the permissions, Plex was able to detect and index the files. However, that’s not best Linux security practice.

The general rule of thumb for Linux permissions is that you should grant sufficient permissions to do what you need but should never grant more lenient permissions than you need to. It turns out, Plex has a very nice article on Linux permissions. Based on that article, the command I should have used for those files was:

sudo chmod 644 -R /plexserver/videos/newfiles

Once I changed the permissions, my Plex server was able to find and index the file.

Technically, what I should probably do is make the user “plex” an owner of the files, which would then allow me to keep the permissions as restrictive as possible. That is done with the “chown” command. If I did that, I could simply give the plex user ownership and then leave the permissions such that the owner can read and write to the file (so, 600). Both of these options actually work, with the first being more permissive than the second. But, you can see with my files that I set one to 644 with me as the owner and another at 600 with plex as the owner:

And, in this screenshot, you can see that Plex was able to recognize them both and index them since it now has “read” permissions for the file:

So, what is the best approach here? If you’re lazy, you could set all the permissions to 777. That gives Plex (and everyone else) complete access to all your media files. If you want to be somewhat restrictive, you could set the permissions for media files in Plex to 644, which gives the owner read/write access and the plex user, if it is not an owner of the file, read access. If you want to be more secure, you could set the permissions to 600 and set the plex user as an owner of the file, which would restrict the permissions for the file to read/write for the owners and no access for everyone else.

 3,608 total views,  6 views today

Plex – Sports Videos Library

Posted on by rcragun

I use Plex to manage my music and video libraries. I actually don’t watch sports very often. The one exception is when I’m grading papers (I’m a college professor). I’ve found that watching soccer/futbol matches in the background while I grade papers breaks up the monotony of grading. With finals coming up, I thought it would be a good idea to record a few games so I had about 10 hours of soccer matches to watch while I grade papers. But I also realized that I wouldn’t remember which matches I had watched, how far into them I was, and I might want to watch some on my TV and others on my computer. Thus, why not put the matches on Plex?

That’s when I realized that Plex doesn’t really have a way to organize sports events. Plex is remarkably good at pulling the relevant information for movies and TV shows (I actually use tinyMediaManager to do the renaming and organizing for movies and TV shows, then put them into the respective Plex folders, which works extremely well). But Plex doesn’t have built-in functionality for recognizing sports events. That, in itself, isn’t a problem. I just need it to recognize the files in a library and keep track of what I’ve watched and what I haven’t. So, here’s how I organized the files.

I created a new top-level folder next to the same folder where I store my movies and TV shows that I called “sports”:

Inside, I created a separate folder for the league I wanted to watch – English Premier League, UEFA Europa League, etc. If I was planning on keeping the games after I watched them, I would have added a folder below that for the year, but I delete the recorded games after I watch them. So, the folder structure looks like this:

I then named the files as follows: [DATE]-[TEAM1 VS TEAM2]. This structure is visible in the above image.

In Plex, click on settings:

Scroll down to “MANAGE” Libraries and click on it:

Click on ADD LIBRARY

In the window that pops up, select “Other Videos.” That lets Plex know that it doesn’t need to try to retrieve metadata for the videos (this is also the option I choose for my home movies):

Name the folder. I called mine “sports”:

Click “Next” then find the top-level folder you created earlier:

Once you’ve found the folder, you’re ready to click ADD LIBRARY.

This tutorial was done using Plex 4.32.1. With that version, you have the option of pinning libraries to your home screen. I went ahead and pinned sports to my home screen so I can see it quickly:

Click on the library and you should see a list of all the sports videos you have in the library:

As you can see in the image above, I had started watching one of the games when I made this tutorial. This is why using Plex to watch the games is so nice – it keeps track of where I was and I can switch between devices.

NOTE: Tutorial created using Plex version 4.32.1.

 6,143 total views,  12 views today

Plex/tinyMediaManager and Doctor Who Specials

Posted on by rcragun

I’m a science fiction fan. And I want my science fiction at my fingertips. To that end, I have slowly been digitizing my favorite series (e.g., Star Trek, Stargate, and now Doctor Who) and putting them on my fileserver that uses Plex to serve the episodes to whatever device I want. I recently ran into an issue with how to organize Doctor Who episodes on my fileserver and, once I figured it out, I thought I’d share it here in case others run into the same situation.

First, props to Doctor Who for occasionally having specials. They are always well done and lots of fun. So, I’m certainly not complaining. BUT, the problem is that the specials are not technically part of a season. That causes some challenges when it comes to how to organize the files.

I use tinyMediaManager for organizing my movies and TV shows on my file server. Not only does tinyMediaManager pull down all the information for my movies and TV shows but it also has the ability to rename the files and organize them. I like how it does all of this and it plays nice with Plex as well, which is important.

Now enter the problem with Doctor Who. Like most TV shows, Doctor Who has seasons. The standard ways to indicate the season and episode of a show are to include something like the following in the name of the digital file: “S03E14.” This indicates the show is from Season (“S”) 03 and is the 14th Episode (“E”) from that season. This works great for most of Doctor Who since most of the content is episodes.

But, what about the specials? The creators of Doctor Who regularly release off-season or out-of-season specials. These don’t get a season episode number. When I tried to get tinyMediaManager to scrape the specials, it didn’t know what to do with them. Basically, it ignored them because it didn’t know what they were since I didn’t know what to name them.

After a little googling, I found out that “Specials” have a special Season designation: “00.” So, for the first Doctor Who (2005) special called “The Christmas Invasion,” that aired on December 25, 2005, how you can name that file is:

Doctor Who – S00E167 – The Christmas Invasion

The naming convention breaks down like this. “S00” tells both Plex and tinyMediaManager that it is a “special.” tinyMediaManager then moves it into a folder called “Specials.” The “E167” is the episode number for the entire series (from Wikipedia). Once I figured this out, tinyMediaManager knew where to put the special:

And Plex began to recognize what the episode was:

Success! Now I can include Specials with my favorite science fiction series. There is order again in the science fiction universe.

 3,540 total views,  10 views today

Plex – Syncing Media to Devices

Posted on by rcragun

I love my Plex server. It works pretty flawlessly with devices like my Roku or allowing me to listen or view my media collection online. However, there are occasionally instances when I know I either won’t have internet access (e.g., on a plane or on a hike) and I want to have access to my media. Plex still has me covered with the option of “syncing” media to my phone. However, the process isn’t all that intuitive. So, here’s my guide to syncing audio and video to a phone/tablet.

First, choosing what to sync is done, from what I can tell, almost exclusively from the device (with some minor exceptions). Basically, you have to install the Plex app on your device, open the Plex app, log into your account on your device, then find the media you want to sync. (I’m not going to cover install, opening, or logging in with this tutorial, just synching.)

Let’s say I want to synchronize my Simon & Garfunkel albums to my phone so I can play their music when I don’t have internet access. I first have to open the Plex app:

This is how the app looks once it’s open and connected to your Plex server.

Make sure it has detected my Plex server (which is called NewPlex):

I circled my server in this image.

Then I have to find my collection of Simon & Garfunkel music in the Music tab (at the bottom of the screen):

To move between different collections, you click the buttons at the bottom of the app. In this screenshot, I was looking at my Music collection.
I have now selected my Simon & Garfunkel songs.

Once I have done that, syncing is pretty easy. Look for the “down arrow” and touch it:

When you do, you’ll get this screen:

You set your synching options on this screen.

You’ll have the option of adjusting the audio quality (this makes a lot of sense if you’ve got your files stored in FLAC on your file server but want to save space on your phone; if so, convert them). You can also dictate how many songs you want to synchronize. If you put a limit, I’m assuming Plex will just randomly select which ones will syncronize (you can also just download specific songs or albums). When you’ve made your decisions, you click on Save and the transfer process will start.

Here’s where things get kind of interesting. You can now open your Plex dashboard and look for your sync progress. To illustrate, I picked an album I have stored in FLAC on my server and started the conversion process:

Your “Conversions” page in the Plex server settings shows you the files that are being prepared to be transferred to your device.

Once your server has converted everything, it will be sent to your device and you’ll see a little arrow in the corner of synchronized albums, like this:

All the white arrows in circles indicate that these albums have been synchronized with my device – so they are stored on my device.

The process is similar for movies. Click on the “Movies” tab at the bottom of the screen to look through your movies. Once you find the one you want to syncronize, click on it. That same “down arrow” will show up:

Click on the down arrow to synchronize the movie to your device.

Click on it and you’ll get one of the same options as before to choose the quality of the synchronized file.

Once you click Save, the conversion process will start on your Plex server:

Here’s the Conversion screen showing my movie converting before transfer to my phone.

Once the conversion is complete, the file will be sent to your device and you’ll see, similarly, an arrow on the movie indicating it is synchronized with your device.

Finally, there are a few secrets to accessing your synchronized content quickly and easily. If/When you find yourself without internet access, or if you just want to see what you have synchronized with your device, you can click on your icon at the top left of the app and select “Offline Browsing”:

Once you do that, only synchronized content will show up in your tabs, like you see below:

You can also click on the “Downloads & Sync” option in that same menu and see the content that is currently synchronized with your device:

And you can check in the settings dashboard in your Plex server to see what content is synchronized with which device (you can synchronize with multiple devices):

This makes it easy to see what content is synchronized to which device

One last bit of advice. If you want to remove content from a device, you can do so on your phone/tablet or on your Plex server. On your phone, go to your Downloads & Sync list, then select the sync you want to delete. When you do, in the upper right corner there will be a trashcan. Click on it and you can delete the sync.

On the Plex server, in the “Sync” tab, hover over the sync and you’ll see a red X. Click on it, and the synchronized files will be deleted from that device

And if you want to remove a device from your account altogether and get rid of all the synchronizations tied to that device, that is done on the Plex server web interface as well. In the Settings menu, click on Authorized Devices, find the device you want to remove and click on the red X.

That will remove all the synchronizations and prevent it from being able to access your Plex server.

This website was helpful in figuring all of this out.

 2,110 total views,  12 views today

Plex – How To Create Smart (auto updating) Music Playlists

Posted on by rcragun

I love my Plex server. I stream my music basically around the world (I travel regularly). But it took me a while to figure out how to create playlists that automatically update (i.e., “smart playlists”) based on search filters. At home, I use a native music client (e.g., Strawberry or Clementine) to play my music. Clementine, in particular, has excellent smart playlist functionality, allowing me to create a playlist with all of my music in the “Folk” genre or published between 1970 and 1980. With how amazing Plex is, I figured the same functionality had to be possible. I knew I could simply add songs to a playlist manually, but that seemed cumbersome. I’d rather let the software do it for me. Once I figured it out, I thought I’d make the steps clear for others.

Counter-intuitively, the place to start is not in the “Playlists” option on the Plex dashboard, but in the Music panel.

Click on the Music pane, not the Playlists pane.

You’d think that you would create Playlists in the Playlists area, but you don’t. You create all the Playlists in the Music area.

Once you’re viewing your music, you need to look for a drop-down menu. It’s all the way to the left and says “All” with a little arrow next to it.

Click on the little arrow next to “All” to drop down the menu.

What you want to do is click on “Custom Filter.” That will open this option:

Here’s where you create custom filters.

Using that filter option, you can search for, say, all the music in the Genre “Alternative” or “Classic Rock.” Once you’ve entered your search criteria, click the “APPLY” button on the far right and it will apply your search criteria to your music:

Once you apply the search criteria, Plex will show you the music that fits the criteria.

Now, creating a “Smart Playlist” is just one more step. Look to the right of the window for an icon with four little lines and a plus sign. That is the icon for creating a playlist:

This is the icon you want for creating a playlist.

Clicking that button will open a prompt for you to name the new playlist. I typically name mine based on the search criteria, but you can call them whatever you want:

Now, with your Smart Playlist created, you’ll be able to see it in the Playlist area. Click on “Playlists” in the left menu and, assuming you’ve done everything correctly, you’ll see your newly created playlist there:

All of your playlists are in the Playlists area.

The little gear icon that appears in the top left corner of the playlist indicates that it is a “smart playlist” that will automatically update if you add new music to your library that meet your search criteria. Playing the playlist is as easy as hovering over it and then clicking on the play arrow that appears:

Hover over the playlist, then click on the play arrow to play it.

There you go. You can now create as many playlists as you’d like using filtering/search criteria. (This guide helped me figure this out.)

 10,629 total views,  46 views today

Building My Own NAS (with Plex, Crashplan, NFS file sharing, bitTorrent, etc.)

Posted on by rcragun

NOTE: As of 2020-06-22, I have a new NAS build. You can read about it here.

For about the last seven years (since 2012), I’ve been using a Synology NAS (Network Attached Storage) device in my house as a central repository for files, photos, music, and movies. It has generally worked well. However, there have been a number of serious problems with the Synology NAS I bought (DS413J). First, the amount of RAM is limited and cannot be upgraded. Second, the CPU (MARVELL Kirkwood, Arm processor) is in the same situation. While the box is small and draws very little power, the inability to upgrade the hardware (other than hard drives), means that I’m basically stuck with what was considered cutting edge back in 2012 when I bought it. In practical terms, these limitations have meant that I have not been able to run Crashplan on my Synology box since the first year I bought it because I have more than a terabyte of files I am backing up and the 512 mb of memory cannot handle that (I created a workaround where I run Crashplan on a different computer, but am backing up the files on the Synology box over the network). It also means that I haven’t been able to run the latest version of Plex for the last 4 years because Plex stopped supporting the CPU in my Synology box. This eventually came to a head about a month ago when the latest Plex client on my Roku stopped working with the very outdated version of Plex server on my Synology NAS device. As a result, I was no longer able to serve videos to my Roku, which was one of the primary reasons why I even have a NAS in the house.

The convenience of having a pre-built NAS with a web interface has been nice. There is a lot to like about Synology products. However, you are locked into their hardware and their software and are restricted by their timelines for upgrading to the latest software. Additionally, my Synology NAS, which is a 4-bay device, has a problem with one of the bays that actually ended up destroying 2 hard drives, so I only have 3 usable hard drive bays. And, Synology devices are crazy expensive. Given my use scenario, paying as much as they now charge for a high-end NAS that might temporarily meet my needs doesn’t make a lot of sense.

So, I finally decided that it’s time to go back to building my own NAS (I had one for a short while before). As I started researching what I wanted for my Do It Yourself (DIY) NAS, I basically went down a rabbit hole of options: Which Operating System (OS)? What hard drives? What file system for the hard drives? Do I use a RAID? What software do I need to install? What CPU and motherboard? How much memory? In this rather lengthy post, I will detail what I ultimately decided to do and why.

DIY NAS: OS (Operating System) Options

As a long-time Linux user, I was never going to consider anything but a Unix-based system. That means I never even considered Windows as an option. Those who use Windows could certainly consider it, but I have no interest in using Windows for my NAS. That, however, didn’t narrow my options that much. With Unix-based systems, all of the following are real contenders: Unraid, FreeNAS, Amahi, Open Media Vault, Rockstor, Openfiler, NAS4Free, or just the Linux distribution of my choice (currently, Kubuntu 18.10). I spent quite a bit of time considering these OSes, all but the last being designed specifically as OSes for NAS boxes. The more I thought about theses pre-packaged options, the more I realized that they fall in between the proprietary OS of my Synology box and the OS I use on all my other computers (Kubuntu), and they are all crippled (to varying degrees) by the same problem I had with Synology – I am beholden to the companies/people who maintain this software to release new packages for the software I need to run: Crashplan and Plex. This is a particular concern with Amahi, Rockstor, Openfiler, and NAS4Free, since they all require the installation of software through their “packages.” That would not be the case if I just went with a standard Linux distribution that gets regular updates (e.g., Kubuntu). I can install pretty much whatever I want on such an OS, which means the NAS will be whatever I want it to be.

Unraid (which isn’t free) will really allow you to install an OS on top of their OS, so that might not be a problem. But I’m also not convinced that I need Unraid for drive management (as I’ll detail below when I discuss how I organized my hard drives). FREENAS is probably the most appealing OS for this as it really is just an OS, and you can install what you want on top of it. My biggest concern here is that FreeNAS is BSD based, which really shouldn’t be a concern, but I have limited experience with BSD (tons with Linux), and I wasn’t certain what FreeNAS would give me over a standard distribution.

There very well may be advantages to one of these specific OSes for NAS devices that I am missing. But, after having suffered under the proprietary lock-in and inability to upgrade my software under Synology, I realized that I was very wary of getting locked into a pre-packaged OS that would mean I couldn’t install what I want to install. Ultimately, I decided to just install Kubuntu 18.04 on what would become my new, DIY NAS box. Some might note that I should have just gone with Ubuntu server, as it reduces the amount of CPU power and memory since I wouldn’t have a graphical front end (KDE). I considered it. But that would also mean that I would have to manage the entire device through the command line or try to find some other software (e.g., webmin, which won’t do everything I need) that would allow me to monitor my device via a web browser. Since I’m most comfortable with a graphical desktop environment, why not just go with what I know and what works for me?

Final choice on OS: Kubuntu 18.04, which is a long-term support release (important for future upgrades to the OS).

DIY NAS: Hardware Options (CPU, motherboard, RAM, etc.)

When Plex stopped working with our Roku, my wife quickly realized it. We all use Plex on a regular basis to watch our movie and TV collection. When I told her what the problem was and said that I thought I was going to need to replace our NAS, she asked me how much it was going to be. Being honest, I told her it could be fairly expensive, depending on what I decided to build. Luckily, we are in a position financially where I could spend upwards of a $1,000 on a new NAS if I needed to and she said that would be fine.

I spent quite a bit of time consider hardware. The biggest question was really whether I wanted to go with dual Xeon processors to build out a real server or go with what I am most familiar, AMD CPUs like Ryzen (I build all my own desktop computers). The dual Xeon approach made a lot of sense as they can manage a lot of concurrent threads. The latest processors from AMD (and Intel, but I’m an AMD guy), Threadripper, also manage a lot of simultaneous processors. However, these are all pretty expensive processors, even somewhat older Xeon’s can be a little pricey if I’m trying to get something that was made in the last few years. Additionally, the motherboards that go with these can be almost as expensive. I debated these options for quite a while.

Then, I had an idea. I had an older computer lying around from my latest upgrade (I usually upgrade my desktop, give my immediate past desktop to my wife; during the last one, I ended up upgrading the entire device, case included, which left a third desktop computer sitting unused). I wanted to test out Plex and Crashplan on that computer just to make sure it worked. It’s an older AMD Athlon II X4 620 that has 24 gb of RAM and room for 8 SATA devices on the motherboard. Once I got everything up and running on this test system, I realized that, given my use scenario, I actually didn’t need the latest Threadripper AMD CPU or even dual Xeon processors. I don’t need concurrent ripping of 4k or 8k video files. I don’t even have any 4k video files (my main TV has 4k capabilities, but I rarely use them). Most of my video is 1080p, which looks great. I tested the system out streaming a video to my TV and backing up files and it worked great. So, I decided to re-purpose this older computer and make it my new NAS.

Once I got everything set up (see below for details), I wanted to see just how much my NAS could handle, especially since the old Synology NAS struggled with streaming just a single HD stream. To run my real world test, I started streaming audio from three Amazon Echo devices in three different rooms in the house via Plex, started streaming a 1080p video file to my phone on the home network, and started streaming a 1080p video file to my main TV. This screenshot shows the Plex server with 5 simultaneous streams:

Simultaneously streaming to five devices on the home network.

The big question was whether my 4 cores could handle this. Here’s how things looked:

All four CPUs were never maxed out simultaneously.

Conclusion: My older AMD Athlon X4 620 was more than up to the task. With two simultaneous 1080p video streams and three mp3 streams, the server was working, but it was far from maxing out. Since we have just one TV in the house, the odds of us ever needing to simultaneously stream more than two videos are almost zero, and that’s true even considering that I am allowing some of my siblings access to my Plex server.

What does this mean for the average person building a DIY NAS? Unless you have, let’s say, 10 4k TVs in your house and you want to simultaneously stream 4k videos to all of them, you probably don’t need the latest and greatest CPU in your NAS. In all likelihood, I’ll let the system I have run for a year or two (unless there is a problem), and then upgrade my box, my power supply, my motherboard, and my CPU (which will also require upgrading my RAM). By that time, Ryzen Threadrippers will have dropped in price enough that it won’t cost me $2,000 to build a beast of a NAS that can serve multiple 4K streams at the same time. I’m not sure I’ll ever need that much bandwidth or power given my use scenario, but I can imagine needing a bit more power in the future.

Where you probably do not want to skimp is on RAM. As I have been transferring my files and media to my new NAS from the old one, I have seen my RAM usage go up to as high as about 16 gb at different times. That has been the result of large bandwidth file transfers, Plex indexing the media files (video and audio), and simultaneous streaming of files. In short, you can go with an older, not crazy expensive, multi-core CPU for your NAS and be fine, but make sure you’ve got at least 16 gb of RAM, maybe more.

DIY NAS: Hard Drive Options

Where I got the most bogged down in my research was in deciding on how many hard drives to use, what file system to use, and whether or not I should use a RAID for the drives. In my Synology box, I had just two 4 terabyte hard drives in a RAID 1 (or mirror) arrangement. While I am getting close to filling up the 4 terabytes (I have about 3 terabytes of photos, movies, files, music), I was more concerned with not losing data. With a RAID 1, all of my data was backed up between the two hard drives. Thus, if I lost a drive, I would still have a copy of all of the data.

Additionally, I pay for unlimited storage through Crashplan. Everything on my NAS is backed up off-site. This way, I have a local copy of all of my files (the mirrored drive) and an off-site copy of all my files (in case there is a fire or catastrophic failure). Since I do back up everything off-site, I could theoretically just go for speed with, for instance, a RAID 0 that stripes all my data between drives. But a failure would mean I would lose all my local data and I would have to restore from the off-site backup (which would take quite some time given the amount of data I have).

As I considered the options for hard drives, my initial thought was to use the same system but increase the size of the hard drives. There are now hard drives with capacities in excess of 10 tb. They are expensive, but two 10 tb hard drives in a RAID 1 would basically replicate what I was doing but give me plenty of room to grow my photo, video, and file collection. I was just about to pull the trigger on this plan until I realized that another option might make more sense.

RAID 6 requires 4 hard drives, but offers a number of advantages over RAID 1 (e.g., similar speed but better redundancy, as well as the possibility of hot swapping hard drives). And, if I went with RAID 6, I could actually buy cheaper, 4 or 6 tb hard drives instead of 10 tb hard drives and end up with just as much or more space as I would get with two 10 tb hard drives for less money. From the many articles I read on this, it seems that lots of corporations use a RAID 6 given the redundancy and speed advantages that result. I ultimately decided to go with a RAID 6 with four 4 tb hard drives. Effectively, that meant I would be doubling my storage (8 tb) while improving my redundancy.

I still needed to figure out what file system to use. Having worked with Linux for over a decade, I typically use EXT4 on all of my computers. It doesn’t have any file size limitations, obviously, and does everything I need. However, I had been hearing about ZFS for a while (as well as BTRFS) and what I had heard made me think that ZFS was really what I should be running on my NAS. While it may slow down my NAS a little bit, the benefits to preventing bit rot and the redundancy it includes meant the impact on performance would likely be worth it. However, ZFS doesn’t come as a standard file system option in Linux distributions. I have used various disk partitioning programs enough to know what the standard file systems are that ship with Linux distributions and ZFS is not one of those. I was a little worried about what might be entailed in installing ZFS and setting up ZFS in a RAID6 (in ZFS terms, it’s called RAIDZ2). Once I found this handy guide, I realized that it wasn’t that difficult and was something that I could easily do. Before I headed down this path, I tested the guide with a spare drive I had lying around and it really was simple to set up ZFS as the file system on a drive. That test convinced me that ZFS was going to be the route to go for my file system on my drives. Thus, my four 4 tb RAID6 is actually a RAIDZ2 with a functional capacity of 8 tb of storage, but is actually expandable should I want to do so. (Also useful for ZFS is this post on scheduling regular scrubs on the drives.)

Update: 2019-08-18 – I restarted my file server after installing system updates and my ZFS pool was missing. That was terrifying. I finally found a solution that was a little nerve-wracking but worked. Somehow, the mount point where my ZFS pool was supposed to mount either got corrupted or had something in it (for me, it was /ZFSNAS). I renamed that folder:

mv /ZFSNAS /ZFSNAS-temp

I was then able to import my ZFS pool with the following command:

sudo zpool import ZFSNAS

Apparently, this is the result of an upgrade to the ZFS software. This happened again right after I rebooted. There is a new option in ZFS systems that has to be enabled in order for these to mount after reboots:

systemctl enable zfs-import.target

Now, when I reboot, my ZFS pool comes right back up.

Update: 2019-11-13 – I installed an updated and restart and now my ZFS pool has disappeared again. The solution above brought it back, but enabling zfs-import.target, doesn’t bring it back up anymore on the next reboot. I tried enabling all of the following (from here):

sudo systemctl enable zfs.target
sudo systemctl enable zfs-import-cache
sudo systemctl enable zfs-mount
sudo systemctl enable zfs-import.target

None of those worked. Not sure what is going on but I’m pretty sure it’s tied to ZFS changes in the latest Ubuntu/Debian kernel. Argh! I also had to do the following after I brought the pool back up to make sure it was shared across the network:

sudo systemctl restart nfs-kernel-server

NOTE: It also looks like I set up my ZFS pool in a problematic fashion. I created one pool, but no datasets whereas I should have had one pool with multiple datasets. Datasets are where the files should be stored, not in the pool directly. I’m now struggling to set up automated snapshots.

DIY NAS: What software to install?

I have been running Plex for about 6 years to manage my media collection. With my Synology box, the limited RAM and not very good CPU meant that it actually had a pretty hard time managing my media collection. It would transmit videos across my network to my ROKU device, but only if they were in a specific format (mp4, which was probably a limitation of the Synology box as that is not a problem with Plex or ROKU). Setting up a video slide show was basically impossible on my ROKU from my Synology box as the CPU and RAM just couldn’t cut it. While I could play my music across the network, the Synology box would also not play nice with my music in Plex. As a result, I used Plex just to watch videos but not for anything else, even though it is a great way to manage all sorts of media – video, photos, and music. Thus, one of the requirements of my new DIY NAS would be that it has to take advantage of all the features that Plex has to offer (I have a lifetime Plex Pass). So, Plex was the core requirement for my NAS.

But I also wanted my NAS to run Crashplan. As noted above, my Synology box didn’t have enough RAM to run Crashplan, which meant I had to run it on my desktop to backup the files on my NAS to Crashplan. It was a hack to get around the limitations of the Synology NAS (FYI, you need about a gigabyte of RAM for every terabyte of files you want to back up to Crashplan). Plex and Crashplan were minimum software requirements. My DIY NAS had to be able to run both of those and run them well.

I do occasionally download stuff via bitTorrent (mostly Linux distributions). So, having a bitTorrent client installed would be nice. Kubuntu comes with one, kTorrent, which was fine.

The last piece of software I really needed was a way to control my NAS remotely. The goal was to basically have it run headless, stick it in a corner in my office, and just let it do its thing. I can control the Plex server through the Plex website, but to do everything else I would need VNC software. I was actually surprised at how difficult it was to find software that would let me control my NAS. I tried Remmina first and had no luck. The interface was clunky, not intuitive, and I only was able to successfully connect about 1 time out of 10. I finally went with nomachine. (I initially went with TeamViewer, but they claimed I was using their software for commercial purposes and cut me off. Fuck them.) Also, in order to make this work reasonably well, you need to install a dummy driver for the X11 driver and create an X configuration file for the dummy driver, as detailed here. Otherwise, the VNC software will be slow as molasses.

I also installed SSH so I could access the NAS remotely in case the GUI VNC programs were having issues. I followed this guide.

(Update: 12/7/2019. I strongly encourage the use of tinyMediaManager for managing your videos and TV shows. It’s the slickest software I’ve found for doing this and runs great on Linux.)

DIY NAS: NFS Fileserver

This guide made it very easy.

DIY NAS: Specifications

Here’s the final rundown of what I put together:
OS: Kubuntu 18.04 (long term support)
Hard Drives: ZFS filesystem; four 4 terabyte drives in a RAIDZ2 for 8 tb of storage
CPU: AMD Athlon X4 620
Motherboard:
RAM: 24 GB
Software: Plex, Crashplan, NFS, TeamViewer, kTorrent

UPDATE – 2019-04-24

It’s been almost 4 months running this NAS box. Generally, it’s worked really well. I have, however, run into two problems.

The first one was pretty recent (about 2 weeks ago). I’m not sure what happened, but over the last month, I’m assuming some update to Kubuntu 18.04 led to the screen going black when I would VNC into the box. The server was still working, but I couldn’t get it to display anything via VNC until I restarted the box manually (SSH on the NAS box would have given me another option for that). I’m still not 100% sure what the problem was, but it was somehow related to KDE. I ended up installing Unity/Gnome as the desktop environment and now the problem is gone. That was about a week ago that I solved the problem. The box has been running without a hitch since then.

The only other issue I will note is that there have been two times when I have been streaming shows on my NAS box through ROKU where my box had to actually transcode a 1080p file. With the hardware I have inside it, the 4-core processor was not up to the task. It ended up stopping playback several times and buffering a lot. It can easily stream a 1080p file in almost any format (both Plex and Roku can handle almost every format) and can even manage to stream two of them simultaneously (tested on my TV and phone simultaneously), but the two movies it had issues with were in a weird MKV format that my Roku couldn’t handle. I ended up ripping them into a different MKV format and, voila, problem solved. The point being – my NAS box is a little underpowered. But, a year or two from now, the price of an AMD Threadripper will be half of what it is now. I’ll swap that in and will have all the power I need.

So, 4 months into this, it’s working extremely well. I’d give it a 99 out of 100.

Update 06-21-2020:

As I predicted, after about 1 1/2 years, it was time to upgrade my NAS. I started a new post detailing my new build and what I changed here.

 6,200 total views,  15 views today