LibreOffice Calc – Comparing and Aligning Two Lists Using VLOOKUP

One of the tasks I have to do regularly as part of my job is to compare two lists to see which items are missing on one list but not the other. I have been doing this by hand but figured there had to be a way to do this in Excel. I finally figured it out but don’t want to forget how to do it. So, I’m documenting it here so I can draw on this whenever I need to do it again.

Here’s the scenario. I have a list of “items” but every month or so I receive an updated list of “items” from someone else. During that month, some of the items on my list have been taken off and some have been added to it. Likewise, the same has happened to the other list that this other person sends me (I’m being really vague here because I work at a university and there are laws that govern academic records).

My list and the other list have to be kept synchronized but we have two separate databases that are used for this because… (argh, yeah). Anyway, what that means is that I have to compare the two lists and quickly find the items on one list but not the other and vice versa. Here’s how to do it quickly in LibreOffice Calc.

Here are two lists:

(NOTE: You can actually do this in two separate Calc sheets or within the same one.)

LIST 1 is in Column A and List 2 is in Column C.

In Column B I am going to create a function that allows me to search all of Column C to see which of the items in LIST 1 show up in LIST 2. Click in Row 2 of Column B then click on the Function Wizard (or start typing):

In the Function Wizard dialogue box, you’ll see this:

Search for VLOOKUP then double-click it and you’ll see this:

You now need to build the function. This is where I got confused, so I’m going to try to explain this carefully.

The first box is the “Search criterion.” Basically, this is what you want to find. In our example, let’s say that we want to see which items in List 1 are in List 2 (Hint: It’s apples and grapes.). So, we are going to put in the Search criterion that we want the software to search the items in column A or LIST 1. We do this by simply selecting cell A2 (you can do this by typing it in or by selecting it by clicking the button to the right of the box and then selecting A2 like I did below):

Next is the “Array” box. This is the content through which you want the software to search to find the items in LIST 1. Again, you can type this in or select it using the button to the right. However, there is an important change that you need to make here. If you select the array with your mouse and leave it as is, the values will change as you drag this formula down in Column B as it will assume you want to adjust the array as well. Since we don’t want to adjust the array but rather want to search through the same items in LIST 2, we need to put dollar signs before the letters and numbers in the Array which will lock the boundaries of the array into place so they don’t change when we copy the formula to other cells, as shown below:

The next part is the part that threw me off for a long time in figuring this out. The “Index” is the column in the Array you want to compare to the Search Criterion. In this case, all you need to do is specify “1” since there is only one column. But, presumably, you could have an array made up of multiple columns and want to choose just the 4th column (so you would enter 4 in the Index). Here’s how this looks:

Finally, the last box that is part of our function is the “Sort order” box. This tells the software whether the lists are ordered alphabetically or not. If they are not, enter zero (“0”). If they are, enter “1.” If you leave this blank, LibreOffice Calc thinks the lists are ordered. So, make sure you fill this out.

Once you’ve got that all entered, your formula should look like what I have above. Hit “OK” and it will search the Array C2:C7 for “bananas.” If it finds it, it will return “bananas.” If it doesn’t, it will return “#N/A” as shown below:

You can then, of course, drag this formula down. When it finds the item from LIST 1 in LIST 2 it returns that item. When it doesn’t, you get #N/A.

And there you have it. A LibreOffice Calc function for searching a list for a target and returning an indicator.

(NOTE: What I typically do after I have run this function is sort by the column where the function is located so I know which items are missing from LIST 1 and which are missing from LIST 2.)

Linux – Adding and Organizing Music with Clementine

In the almost 15 years I’ve been using Linux I have gone through a number of music playing apps. From Amarok to Banshee to Rhythmbox and at least a few more. My favorite at the moment is Clementine. I’ve grown to really like the basic interface of Clementine and it does a good job with pretty much everything I need.

The one issue I had with it over the years is that I didn’t think I could use it to add music to my music library and organize it at the same time. Turns out, I was wrong. Clementine does have the ability to add songs to your music library and organize them (into folders and renaming the files). It’s just not the most intuitive process. So, I figured I’d outline the process below.

First, you obviously need to install Clementine.

You’ll then need to set your music directory in Clementine which is where you want to store your music files. Go to Tools -> Preferences.

In the window that opens, click on “Music Library” and then add the folder where you want to store your music:

Next, to add music to your folder, go to the left side of Clementine and look for where it says “Files”:

Click on that and you’ll be able to browse to any folder on your computer.

I usually put the music I want to add to my collection on my desktop, so I navigate there, find the folder, and then right-click on it. In the context menu that comes up, click on “Copy to library.”

In the window that comes up next, you’ll have the option to change how the files are organized, both at the folder level and with the renaming of files.

You also have the option of changing the destination (which doesn’t make much sense since you want the music in your primary music folder). You also have a bunch of other options, like deleting the files after they are copied, changing the naming conventions, etc. Once you’re ready, hit “OK” and your music files will be copied into your music library named exactly how you want them to be named and organized how you want them to be organized.

Linux – Fixing the Epson (V33 Perfection) Scanner Issue in Kubuntu 18.04

As of Kubuntu 17.10, my Epson V33 Perfection scanner stopped working with Linux. I bought this scanner specifically because it did work with Linux. That it stopped working was a serious disappointment as I have been using the same scanner for years and regularly need it for a variety of things.

I tried installing older versions of the software (I’ve kept the downloads from Epson for years) but that didn’t work. I tried it on a different computer with the same OS and that didn’t work. I tried a lot of troubleshooting and came up empty. After hours of trying various solutions, I gave up (that was a few months ago).

However, when my CPU died on my desktop computer forcing me to reformat it again about a month after I had previously tried, I figured it was time to try to solve this again. This time, however, I quickly came across the answer online (see here). It turns out, this is a bug in the latest version of the scanning software that ships with Linux, which confirmed that it wasn’t a problem with my scanner or with Epson’s software. Here’s what I did to fix the problem.

First, go ahead and install the latest version of Epson’s software from their website (start here – or just here – lots of clicks to get to where you need to go).

Untar the file you download and read the directions for how to install the software. (Basically, navigate to the directory you just untarred in a terminal then run ./install.sh).

Before this bug, that was all you needed to do and the scanner would work. Now, there are two more steps.

The second step, also at the terminal, is to type in:

sudo ln -sfr /usr/lib/sane/libsane-epkowa* /usr/lib/x86_64-linux-gnu/sane

I’m not exactly sure what this command does but the “ln” command creates symbolic links, so I’m guessing it’s linking two files or commands that help correct the bug in the latest iteration of the software.

Then, open a text editing program and create a new, blank file. In that file, you’re going to add the following content:

# chmod device EPSON group
ATTRS{manufacturer}=="EPSON", DRIVERS=="usb", SUBSYSTEMS=="usb", ATTRS{idVendor}=="04b8", ATTRS{idProduct}=="*", MODE="0777"

The first line of this file is just a comment. The second adds information about Epson scanners.

Save the file as “79-udev-epson.rules”.

Now, move that file from wherever you created it to the following directory:

“/etc/udev/rules.d/”

Like this:

Now, restart your computer. This worked for me. Hopefully, it will work for you too.

 

Linux: How to Install Zotero Standalone (in KDE or Kubuntu)

I’ve used Zotero to manage my collection of books, articles, and other citations since Zotero was released. The software has become much more robust over time. As of the latest major milestone release (5.0), Zotero has moved to standalone software exclusively (they no longer piggyback on Firefox). With Mac and Windows, you can download executable files to install Zotero. With Linux, it’s a little trickier. Here’s my method for installing Zotero so I can launch it from the KDE application menu.

First, download the tar.gz file from Zotero.

Second, untar (i.e., unzip) the folder inside the tar.gz compressed file.

Move the resulting folder (Zotero_linux-x86_64) to a directory where you’ll want to run Zotero from. I use my /home/[user]/ folder but it can be any folder with read/write/execute access.

The Zotero website says that you should then run the bundled “set_launcher_icon.” I usually do that, but I still need to add Zotero to my application launcher. To do this, right-click on the KDE application launcher menu like this:

Select “Edit Applications.”

In the window that pops up, decide where you want to put Zotero (I went with Education) then select “New Item.” You’ll get this window:

Fill it in with the following information:

Name: Zotero
Description: (Whatever you want.)
Comment: (Again, whatever you want.)
Command: /home/[user]/Zotero_linux-x86_64/zotero

You can also add an icon for Zotero (I always do), like this one.

Obviously, replace [user] with your username. The line that really matters is the Command line as that is where the OS will know to look for the Zotero launcher. Make sure it is pointing to the file names “zotero” in the application folder (not “zotero.desktop” as that won’t launch the application).

Once you’ve done that, you should be able to launch Zotero for the quick launcher or from the application launcher:

 

 

Virtual Private Hosting – How to Remove a Domain with Certbot SSL Certificate (on Ubuntu 16.04 with apache)

It used to be pretty easy to remove a domain from my Virtual Private Server – dissable the site in apache, delete the files, delete the underlying database, and remove the domain from my DNS manager. Done.

With SSL certificates now a standard part of hosting websites (see here), this is complicated by the removal of those certificates. I am extremely grateful to the EFF for providing a free way to get SSL certificates through certbot and letsencrypt. Adding domains with certbot is pretty easy, but changing the domains with certificates is not so easy. In fact, it’s pretty complicated and there aren’t great directions out there (which is why I put this guide together).

In this guide, I’ll show you how to remove a domain from a virtual private server (I use linode.com) along with removing the SSL certificate. I was hosting a domain and website for a friend, but that person decided they no longer needed the domain. Here’s how I removed it.

First, it’s always a good idea to back up the files associated with the domain, just in case. I use phpMyAdmin to manage my SQL databases. Log in to your phpMyAdmin site, find the database associated with your site, and select Export.

You shouldn’t need to change any options, just select “Go” and phpMyAdmin will export the entire database.

To download all the files associated with your site, you can zip them from an SSH terminal or download them via FTP. I use Filezilla. Find the folder that contains all the files for your website and download the whole thing.

Now that you have a backup of everything, it’s time to start unmounting.

First, you should dismount your site in apache.

sudo a2dissite [domain]

You also need to dismount the site with encryption, which is the same command, but with the following addition:

sudo a2dissite [domain]-le-ssl.conf

Then reload your apache2 configuration:

service apache2 reload.

If you’ve done everything correctly, when apache reloads, there won’t be any errors.

Next, delete the certificate associated with your domain in certbot. To delete the certificate associated with a specific domain, the command is:

certbot delete –cert-name [enter domain here]

That will delete the certificate associated with that domain.

You may also need to update your certificate profile by using the command:

certbot –apache

This will list all of the domains you have certificates for. You can then select all of the other domains on your server but drop the one you want to delete. That will create a new certificate with all of the domains minus the one you have deleted.

If you are wondering whether certbot has actually removed your domain, you can check by going to /etc/letsencrypt. You want to make sure that your domain is no longer showing up in either the /live,  /renewal, or /archive folders. If you still see it in there, it should be safe at this point to delete any folders with the name of the domain in it.

Next, you can delete the corresponding database. In phpMyAdmin, click on “Server: localhost” then click on “Databases.” Select the box next to the database you want to delete, then, at the bottom, click “Drop”. You’ll get a warning about destroying a database. Select OK and the database is gone.

We’re almost done. Now, delete the files for your site. You can do this with the rm command from the terminal or using your FTP client.

Now, you should delete the apache site files that are located in /etc/apache2/sites-available. These are the two that were dismounted earlier: [DOMAIN].conf and [DOMAIN]-le-ssl.conf.

You can restart the apache service one more time to make sure everything is working, but you should be good.

Finally, you can delete the domain from your DNS manager.

That should do it. The domain should now be gone, entirely, from your server.

Linux – 360 degree video editing on Linux

I recently purchased a 360-degree camera that I have used a few times. It takes 360-degree spherical photos and also films in 360 degrees. It’s not the most expensive such camera, but it does a decent job shooting 360-degree panoramic videos.

However, I have run into a couple of problems with the resulting footage on Linux.

First, as is the case with pretty much any footage shot on your phone, it can be bumpy and really needs to be stabilized. I addressed this to some degree by purchasing a 3-axis gimbal, which minimizes the need for stabilization. However, the software I typically use on Linux for editing videos – Kdenlive – doesn’t do a great job stabilizing 360-degree video. Additionally, Kdenlive doesn’t have a profile for 360-degree video and simply recognizes it as an equilateral rectangular clip (1920×960, 27.90fps). It’s still possible to edit the video using Kdenlive, but it treats it as just a wide angle clip and not as 360-degree video. Otherwise, Kdenlive edits the video as if it was any normal video clip.

Second, when you render an edited 360-degree video file in Kdenlive, it loses the tag necessary to tell youtube (or a desktop player; see below) that the video is 360-degree footage. As a result, it’s necessary to re-add that tag before the video can be played back on your computer or uploaded to youtube. There is a python script that can do this but it only works under Windows or Mac. It was released by Google and the GUI version is available here. I run it on a virtual machine and it works fine. It would be nice to be able to simply edit the necessary tags in the video using something like VLC or FFMPEG or a command line in Linux, but I have been unable to find directions that explain how to do that at this point.

Third, there are issues with playback. Until just recently (as in the end of 2017), playback of 360-degree videos on Linux was not really possible. There wasn’t a video player that had this option. However, as of the 3.0.0 version of VLC, 360-degree video playback is now possible. (As a bonus, VLC can also open 360-degree images as well. I love VLC.) Of course, VLC 3.0.0 doesn’t ship with most current Linux distributions, so you’ll have to install it from their PPA nightly branch. With VLC 3.0.0, it is now possible to video 360-degree video on Linux.

What is really needed to streamline this process on Linux? If the awesome folks running Kdenlive could create a 360-degree video profile (at least as a rendering option) that includes the necessary spherical tag, that would mean I could skip step 2 above entirely. That would be awesome. I’d happily donate some money to them if they could make that happen.

 

LibreOffice Calc – splitting contents of cells into multiple columns (e.g., splitting commas)

I periodically have to take a column of text in LibreOffice calc that has names like this “Lastname, Firstname” and split them into two columns. I figure it out every time, but then I forget how I did. So, here’s a quick tutorial on how this is done.

Open your spreadsheet with the cells that need to be split, like this:

Select the column that you want to split:

Then go up to Data -> Text to Columns:

You’ll get the following window:

This window gives you several options for splitting the cells, using commas, spaces, semicolons, tabs, other, etc. I selected just “Tab” and “Comma” but could also select “Space” to get rid of the extra space. However, I’m going to leave the extra space and show you one more function that can be useful in more complex situations. Once you’re done, hit “OK” and you’ll see your single columns split into two:

If you want to get rid of the extra space, there is a LibreOffice Calc function for that. Click in cell C1 and then go up to Insert -> Function. You’ll see this window:

The function you want is in the Text Category (use the dropdown menu) and is called TRIM. Simple TRIM the text in B1 and it will get rid of the extra spaces:

When you’re done with your function, select “OK” and you’ve got your spaces removed. Drag that function down and it will remove all of the spaces:

You can then copy the new column without the spaces and do a “Paste Special” into the old column, overwriting the text with the spaces. Just make sure you turn off the “Formulas” when you do the “Paste Special” and you’ll get just the new text:

Delete the column with formulas and you’re good to go:

 

R (Linux) – creating a wordcloud from PDF

On my professional website, I use wordclouds from the text of my publications as the featured images for the posts where I share the publications. I have used a website to generate those wordclouds for quite a while, but I’m trying to learn how to use the R statistical environment and knew that R can generate wordclouds. So, I thought I’d give it a try.

Here are the steps to generating a wordcloud from the text of a PDF using R.

First, in R, install the following four packages: “tm”, “SnowballC”, “wordcloud”, and “readtext”. This is done by typing the following into the R terminal:

install.packages("tm")
install.packages("SnowballC")
install.packages("wordcloud")
install.packages("readtext")

(NOTE: You may need to install the following packages on your Linux system using synaptic or bash before you can install the above packages: r-cran-slam, r-cran-rcurl, r-cran-xml, r-cran-curl, r-cran-rcpp, r-cran-xml2, r-cran-littler, r-cran-rcpp, python-pdftools, python-sip, python-qt4, libpoppler-dev, libpoppler-cpp-dev, libapparmor-dev.)

Next, you need to load those packages into the R environment. This is done by typing the following in the R terminal:

library(tm)
library(SnowballC)
library(wordcloud)
library(readtext)

Before we begin creating the wordcloud, we have to get the text out of the PDF file. To do this, first find out where your “working directory” is. The working directory is where the R environment will be looking for and storing files as it runs. To determine your “working directory,” use the following function:

getwd()

There are no arguments for this function. It will simply return where the R environment is currently looking for and storing files.

You’ll need to put the PDF from which you want to extract data into your working directory or change your working directory to the location of your PDF (technically, you could just include the path, but putting it in your working directory is easier). To change the working directory, use the “setwd()” function. Like this:

setwd("/home/ryan/RWD")

Once you have your PDF in your working directory, you can use the readtext package to extract the text and put it into a variable. You can do that using the following command:

wordbase <- readtext("paper.pdf")

“wordbase” is a variable I’m creating to hold the text from the PDF. The variable is actually a data frame (data.frame) with two columns and one row. The first column is the document ID (e.g., “paper.pdf”); the second column is the extracted text. You can see what kind of variable it is using the command:

print(wordbase)

This gives you the following information:

readtext object consisting of 1 document and 0 docvars.
#  data.frame [1 × 2]
doc_id      text
<chr>        <chr>
1   career.pdf "\"      \"..."

R won’t show you all of the text in the text column as it is likely quite a bit of text. If you want to display all the text (WARNING: It may be a lot of text), you can do so by telling R to display the contents of that cell of the data frame, which is row 1, column 2:

wordbase[1, 2]

“readtext” is the package that extracts the text from the PDF. The readtext package is robust enough to be able to extract text from numerous documents (see here) and is even able to determine what kind of document it is from the file extension; in this case, it recognize that it’s a PDF.

The list can now be converted into a corpus, which is a vector (see here for the different data types in R). To do this, we use the following function:

corp <- Corpus(VectorSource(wordbase))

In essence, we’re creating a new variable, “corp,” by using the Corpus function that calls the VectorSource function and applies it to the list of words in the variable “wordbase.”

We’re close to having the words ready to create the wordcloud, but it’s a good idea to clean up the corpus with several commands from the “tm” package. First, we want to make sure the corpus is a plain text:

corp <- tm_map(corp, PlainTextDocument)

Next, since we don’t want any of the punctuation included in the wordcloud, we remove the punctuation with this function from “tm”:

corp <- tm_map(corp, removePunctuation)

For my wordclouds, I don’t want numbers included. So, use this function to remove the numbers from the corpus:

corp <- tm_map(corp, removeNumbers)

I also want all of my words in lowercase. There is a function for that as well:

corp <- tm_map(corp, tolower)

Finally, I’m not interested in words like “the” or “a”, so I removed all of those words using this function:

corp <- tm_map(corp, removeWords, stopwords(kind = "en"))

At this point, you’re ready to generate the wordcloud. What follows is a wordcloud command, but it will generate the wordcloud in a window and you’ll then have to do a screen capture to turn the wordcloud into an image. Even so, here is the basic command:

wordcloud(corp, max.words = 100, random.order = FALSE)

To explain the command, “wordcloud” is the package and function. “corp” is the corpus containing all the words. The other components of the command are parameters that can, of course, be adjusted. “max.words” can be increased or decreased to reflect the number of words you want to include in your wordcloud. “random.order” should be set to FALSE if you want the more frequently occurring words to be in the center with the less frequently occurring words surrounding them. If you set that parameter to TRUE, the words will be in random order, like this:

There are additional parameters that can be added to the wordcloud command, including a scale parameter (scale) that adjusts the relative sizes of the more and less frequently occurring words, a minimum frequency parameter (min.freq) that will limit the plotted words to only those that occur a certain number of times, a parameter for what proportion of words should be rotated 90 degrees (rot.per). Other parameters are detailed in the wordcloud documentation here.

One of the more important parameters that can be added is color (colors). By default, wordclouds are black letters on a white background. If you want the word color to vary with the frequency, you need to create a variable that details to the wordcloud function how many colors you want and from what color palette. A number of color palettes are pre-defined in R (see here). Here’s a sample command to create a color variable that can be used with the wordcloud package:

color <- brewer.pal(8,"Spectral")

The parameters in the parentheses indicate first, the number of colors desired (8 in the example above), and second, the palette title from the list noted above. Generating the wordcloud with the color palette applied involves adding one more variable to the command:

wordcloud(corp, max.words = 100, min.freq=15, random.order = FALSE, colors = color, scale=c(8, .3))

Finally, if you want to output the wordcloud as an image file, you can adjust the command to generate the wordcloud as, for instance, a PNG file. First, tell R to create the PNG file:

png("wordcloud.png", width=1280,height=800)

The text in quotes is the name of the PNG file to be created. The other two commands indicate the size of the PNG. Then create the wordcloud with the parameters you want:

wordcloud(corp, max.words = 100, random.order = FALSE, colors = color, scale=c(8, .3))

And, finally, pass the wordcloud just created on to the PNG file with this function:


dev.off()

If all goes according to plan, you will have created a PNG file with a wordcloud of your cleaned up corpus of text:

NOTE:

To remove specific words, use the following command (though make sure you have converted all your text to lower case before doing this):

corp <- tm_map(corp, removeWords, c("hello","is","it"))

Or use this series of functions, which is particularly helpful for removing any leftover punctation (e.g., “, /, ‘s, etc.):

toSpace <- content_transformer(function (x , pattern ) gsub(pattern, " ", x))
corp <- tm_map(corp, toSpace, "/")
corp <- tm_map(corp, toSpace, "@")
corp <- tm_map(corp, toSpace, "\\|")

Source Information:

Reading PDF files into R for text mining
Building Wordclouds in R
Word cloud in R
Removing specific words
Text Mining and Word Cloud Fundamentals in R
Basics of Text Mining in R

For Advanced Wordcloud Creation:

There is another package that allows for some more advanced wordcloud creations called “wordcloud2.” It allows for the creation of wordclouds that use images as masks. Currently, the package is having problems if you install from the cran servers, but if you install directly from the github source, it works. Here’s how to do that:

install.packages("devtools")
library(devtools)
devtools::install_github("lchiffon/wordcloud2")
letterCloud(demoFreq,"R")

You can then use the “wordcloud2” package to create all sorts of nifty wordclouds, like this:

Before you can use wordcloud2 to create advanced wordclouds, you need to convert your data (after doing everything above) into a data matrix. Here’s how you do that:

dtm <- TermDocumentMatrix(corp)
m <- as.matrix(dtm)
v <- sort(rowSums(m),decreasing=TRUE)
d <- data.frame(word = names(v),freq=v)

The data matrix is now contained in the variable “d”. To see the words in your frequency list ordered from most frequently used to least frequently used, you can use the following command. The number after “d” is how many words you want to see (e.g., you can see the top 10, 20, 50, 100, etc.)

head(d,100)

To create a wordcloud using wordcloud2, you use the following command:

wordcloud2(d, color = "random-light", backgroundColor = "grey")

And if you want to create a wordcloud using an image mask (the image has to be a PNG file with a transparent background, you use the following command:

wordcloud2(d, figPath = "figure.png", backgroundColor = "black," color = "random-light")

Note: Source for directions on wordcloud2 are here and here; though see here for converting your corpus into a data matrix, which is what you have to use to create these fancy wordclouds.

NOTE: Color options for wordcloud2 are any CSS colors. See here for a complete list.

R (Linux) – basic installation

To install the R programming environment on Linux is pretty straightforward, but it does require a little bit of know how in order to find the correct packages. As is typically the case with Linux, there are multiple ways to get things done. I like to use Synaptic for installing and removing software, but you can also use the software manager that comes with your Linux distribution (in Linux Mint it’s called Software Manager) or the command line (in KDE based distributions, Konsole).

For the most up-to-date installation of R, it’s actually best to install directly from the R repository. A list of Linux repositories for the R environment is located here. In order to install from the repository, you need to update your list of repositories in Synaptic. To access your repository list in Synaptic, click on Settings -> Repositories.

In the new Software Sources window, click on “Additional repositories” and you’ll get this window:

Click on Add a new repository. You’ll get this window:

The exact information you put into that window will vary based on which mirror you chose. Here is what I added in mine:

deb xenial/

In order to ensure you have the right files and to follow best security practices, you should install the signing key as well. Directions for installing the signing key are found here, but it can be done with a simple command from a terminal:

sudo apt-key adv –keyserver keyserver.ubuntu.com –recv-keys E084DAB9

Once you have done all of that, you can install R from Synaptic.

First, open Synaptic, which will require your password. You’ll get the basic Synaptic Package Manager window:

Next, in the search box, search for “r-base”. Right-click it and select “Mark for installation” to install “r-base”:

In the above screenshot, I have already installed r-base, so the option “Mark for installation” is greyed out. But, obviously, that’s what I already did. When you select this, Synaptic will automatically select all the other necessary packages (there are about 10 to 15 additional packages necessary for R to run: r-cran-class, r-cran-lattice, r-cran-spatial, r-cran-survival, r-cran-codetools, r-cran-nnet, r-cran-mass, r-cran-boot, r-cran-nlme, r-cran-rpart, r-cran-cluster, r-cran-kernsmooth, r-cran-foreign, r-cran-mgcv, r-cran-matrix, r-recommended, r-base-core).

If you plan on installing any other R packages, it’s not a bad idea to also install “r-base-dev,” as it helps fill in dependencies for other packages.

Once you’ve selected r-base, hit Apply in Synaptic and all the software will be installed.

You now have the base software for R installed.

To open the R environment in a terminal, launch a terminal and simply type “R” at the prompt, like this:

Here’s where things can get a little complicated. To do different things in R requires various libraries or packages. Some of these can be installed using the R terminal while others need to be installed from your Linux distribution’s repositories. To install a library or package using the R terminal, you use the following command once you have opened the R environment:

install.packages(“PACKAGENAME”)

The first time you run this, the R environment will ask you to select a mirror.

Choose one close to your location. R will then install the package, assuming you type everything correctly.

NOTES:

Before you start trying to install additional R packages, it’s a very good idea to install the following Linux packages:

r-base-dev build-essential

If you run into an error message, there are several possibilities. First, check to make sure you typed everything correctly. R is not forgiving on spelling mistakes. Second, if the error is something like:

installation of package ‘PACKAGENAME’ had non-zero exit status

Or

dependency ‘PACKAGENAME’ is not available

There is a good chance that you need to install a package or library using Synaptic (or from a terminal using apt). For instance, to install the “tm” package, there is an unsatisfied dependency (meaning, a library or package that needs to be installed but cannot be installed using the R installer). The dependency is the ‘slam’ package. This can be installed using Synaptic (or, from a terminal, using the command “sudo apt-get install r-cran-slam”). Once you’ve installed the dependency, try re-installing the package and the error messages should go away.

 

NOTES:

I also have found that I like RStudio as an IDE for working with R. It’s a little bit friendlier to use than a straight command line interface as it keeps track of variables and loaded libraries. The personal version for your desktop can be downloaded here.

And a note on RStudio on Linux. I regularly get an offset from the cursor location and where the cursor actually is in the command window. It turns out this is a font issue. If you go up to Tools -> Global Options -> Appearance and change the font to anything else, this problem will go away.

Kdenlive playhead doesn’t reposition on Linux

I use Kdenlive for editing videos when I have the occasional need to edit videos.  In recent versions of Kdenlive (e.g., 17.07.70), I have been running into a problem that has been driving me crazy.  The line that runs across the timeline editor, which is called the “playhead,” has been freezing in place.  I could click on a location and that location would show up in the Project Monitor one time, but when I tried to click anywhere else in the editor or move the playhead, it wouldn’t move.  It would stay where it was.  This made Kdenlive unusable.

Here’s a video illustrating the problem:

It took forever to find the solution to the problem, and the solution reveals that this is a bug in this version of kdenlive.  Apparently what is causing the problem is having multiple processor threads turned on in the settings in kdenlive.  Go to Settings -> Configure Kdenlive.  You’ll get this window:

Go to the Environment tab and you’ll see this:

If you’re having this problem, you’ve probably got your Processing threads variable set to something other than 1.  I had mine set to 8, thinking the 8 core cpu I have would speed up the editing process for 4k video.  As it notes in the screenshot, any threads above 1 is experimental.  By experimental, I guess the Kdenlive developers really mean “unusable.”

To solve the playhead not being movable problem, set the Processing threads variable to 1 (I tried other values as well, and only 1 works).  Hit OK.

Then restart Kdenlive. You should now be able to drag your playhead around again and edit videos, like this: