Backing Up Files to Prevent Data Loss

Posted on by rcragun

It took years to figure this out and it’s not free, but I don’t ever lose files anymore. Not losing files is worth the $240 I pay for these services per year.

Do note that I use Linux as my operating system, which is part of the reason why I chose the software that I use to back up my files, but the services I use run on Windows and Mac as well. Also, I do have some technical know-how, but, honestly, the approach I’m going to recommend is really not that complicated and could be done by pretty much anyone if you spend a little time setting it up.

Active Files vs. Archive Files

One of the most important distinctions I have made over the years when it comes to my files is realizing that some of my files include current or active projects while other files are no longer active and can be archived. Active files are those that relate to current projects I have (I’m a college professor and do a lot of research). Any current research project, any files for classes I teach, and any files I need for service or personal affairs are considered “active.” Active means I am likely to need access to those files sometime in the next 6 months to a year. But, more importantly, I want active files to be accessible instantly on any one of my devices (I have two desktop computers, a laptop, a tablet, and a phone).

Any files that do not meet the criteria for “active” can be archived. Archived files are still relatively accessible with the system I will detail below, but not as readily as are active files. I can access them on my home network or via a website (as I’ll detail below), but they are not stored on all of my devices such that I can immediately access them. It takes a few minutes to access archived files. Examples of archived files are those for projects I have completed, old financial information, old class files, or old service files. Effectively, they are files that I do not foresee needing within the next 6 months to a year, but I could potentially need them at some point in the future, so I am not deleting them. Also included in my archive are photos, home movies, music, and other videos.

Active File Backup

To back up my active files, I use Dropbox. That’s not an affiliate link. I’m not getting any money for recommending Dropbox because I’m not necessarily recommending Dropbox. You could use a different backup program. The reasons I went with Dropbox are: (1) It runs on Linux. (2) It provides me with sufficient storage for my active files. And (3) it synchronizes my files across all my devices while also keeping a copy in the cloud.

Here’s the key to using Dropbox, though: ALL of your active files have to be stored in your Dropbox folder!!! All of them!!!

If you don’t store a file in your Dropbox folder and then your computer crashes and the hard drive is ruined, that file is gone, permanently. But if you store it in Dropbox, the fine folks at Dropbox will upload that to their cloud service and synchronize it to your other computers. Effectively, Dropbox provides real-time backup for your most critical files. I use just the Personal Plus plan, which is roughly $10/month and gives me 2 terabytes of backup. That is sufficient storage for my active files throughout the year. Since my files are my job, $120 per year to make sure I don’t lose any of them is well worth it to me.

Here’s a screenshot showing my Dropbox folder:

As I complete projects or, when the year ends, I archive my photos and other data from the previous year, moving those files out of my Dropbox folder and into my archive.

NOTE: For some people, a Dropbox plan may be sufficient space for both active and archive files. If you don’t have a lot of photos, don’t store any videos, and don’t create a lot of files, 2 terabytes is a lot of space and Dropbox may be all you need. If so, great. You can stop here. Get a Dropbox account and put everything in it and make sure it is running on your computer. Then, if something goes wrong, you’ve always got a backup of your files.

Archive File Backup

If you’re still reading this, I’m assuming it’s because you have a lot of files (or photos or videos) and you are looking for a reliable way to back those up. I use a two-pronged backup approach for my archived files, though the second part is really sufficient for most people.

First, I built a NAS device (NAS = Network Attached Storage). While it is very helpful, it’s actually not necessary depending on your device and how much storage you have. On my NAS, I have a raid with roughly 8 terabytes of storage. The raid is set up so I could lose up to 2 of the 4 hard drives without losing any data. In other words, my archived files are backed up across multiple disks, locally, such that I would not lose any data if one of my hard drives crashed. That is a key if you are doing local backups. But, the second part of the archive means that is also not entirely necessary.

Second, I use CrashPlan for backing up my archive. I originally started using CrashPlan when they had a plan for individual users. They have since switched that to a small business plan that is $10/month. The reasons I chose CrashPlan are pretty simple: (1) It runs on Linux. (2) It runs in real-time without me having to monitor it. (3) It doesn’t limit me to a specific amount that I can back up. (4) It has an option online that allows me access to my archived files, even though it is fairly slow.

Here’s a screenshot showing my NAS mounted on my desktop computer at home:

Why did I suggest above that you don’t need a NAS device to use CrashPlan? If you’ve got lots of files taking up lots of space on your computer, I’m guessing that you either have installed another hard drive in your computer or you have an attached storage device, like a backup hard drive. I can easily envision a scenario in which someone has a Windows computer with a 1 terabyte solid-state drive for their OS and primary programs, games, etc. but also a 10 terabyte disk drive also installed in their computer where they store their photos, videos, music, and other old files that take up a lot of space. On just that one computer, it’s possible to run both Dropbox and CrashPlan (though, you’d need quite a bit of RAM to do so). Dropbox would be used to backup active files and CrashPlan could be set to backup just files on the large disk drive. As a result, a NAS isn’t necessary. I went with a NAS because I want it to always be on, serving up videos and music to my other devices in my home and across the internet. But it is possible to use this approach without a NAS.

Conclusion

In summary, if you have mission-critical files that you need to back up in real-time, I strongly recommend a backup service like Dropbox. Consider those your “active” files and make sure they are always saved in Dropbox. If you want to keep older files that you don’t need on an active basis, create an archive (either a separate folder or a separate drive or even a separate computer) and use CrashPlan or a similar service to back up those files. With such an approach, the odds of you losing important files, including old photos, drops pretty precipitously.

 380 total views,  3 views today

LibreOffice Calc – Conditional Formatting

Posted on by rcragun

I haven’t used this feature of spreadsheet software as much as I probably should, but I have a spreadsheet I have been working with a lot lately and conditional formatting has been key to helping me orient myself in the spreadsheet. However, in the process, I realized that I am really a noob when it comes to conditional formatting. To help get myself up to speed, I created the following tutorial so I can reference it again in the future. (NOTE: I’m using LibreOffice 7.2.4.1 for this tutorial.)

Before I begin, what is conditional formatting? The basic idea is that you can change the look of a cell in a spreadsheet (or many cells) based on the contents of those cells (or other cells) without having to manually change the formatting of the cells. Instead, you “program” into the spreadsheet how you want cells to look assuming they meet specific criteria. Then, when you enter information into your spreadsheet, the cell formatting automatically updates to reflect the information in the cell.

Tutorial A – “Formula Is” Conditional Formatting

Let’s start with a very basic illustration. Let’s say I want to use a spreadsheet as a to-do list. Here’s how my spreadsheet looks, to begin with:

What I want to do is pretty simple. If I add an “x” to the Complete column for an item, I want that item to be crossed out. Effectively, I am going to change the font of the tasks to “strikethrough,” which crosses out the text. This way, I can quickly and easily discern which items are completed on my to-do list and which still need to be completed.

How do I set up the conditional formatting?

First, select the first cell (A2) that says “grocery shopping.” Then go up to “Format” on the menu and select “Conditional” -> “Condition.”

You’ll then get this window

An explanation of the window is in order. At the bottom, the range of cells to which the conditional formatting will apply is included. Obviously, this can be set to a range of cells or just a single cell. Right now, I have it applying to just a single cell, A2.

Where it says “Condition 1” is where you will build your condition statement. If you click on the drop-down arrow where it says “Cell value,” you’ll see a couple of additional options:

Those options are “Formula is” and “Date is.” This tutorial will cover two of the three options. For this one, we’re going to go with “Formula is.” The reason why we are going with “Formula is” is because we are going to use the information in a different cell, B2, to change the formatting in the current cell, A2. As the next tutorial will illustrate, you can also use the information in the same cell (e.g., A2) to change the formatting of that cell. So, select “Formula is.” You’ll notice that the window changes and you now just have a single box next to “Formula is” where you can enter your formula:

For our very simple to-do list, our formula will be pretty simple. We’re going to tell Calc to check to see if there is a lower-case “x” in B2. Here’s the formula: B2=”x”. (NOTE: If you want text or a character as part of your formula, it has to be inside quotes.) Here’s how it looks in the window:

Below that, we need to indicate how we want the formatting to change. If you click next to “Apply Style” where it says “Accent,” you’ll see a list of built-in formatting options:

Since we want to have the text crossed out and that isn’t one of the built-in options, we’re going to select “New Style”. We’ll then get this window:

We actually only need to change two things. First, name the style. Since we are going with “Strikethrough,” let’s call this “Strikethrough.”

Next, Strikethrough is a Font Effect, so we click on the Font Effects tab and look halfway down the screen to where it says “Strikethrough.” By default, it says “(Without).” Click on that drop-down and select “Single” (or “Double” if that’s your fancy).

When you’re done, click “OK” and you’ll be taken back to the original Conditional Formatting window.

That’s actually all we need to do for now. Click OK on that window, and let’s test our Conditional Formatting. To test it, type a lower case “x” in B2 and hit “Enter” to see if your formatting works.

Okay, one more thing and the first part of this tutorial is done. We applied that conditional formatting to just one cell, A2. But we want it to apply to all the cells in that list. How do we copy the formatting and apply it to the others instead of having to repeat the process above? Select A2 and right-click. You’ll get a context menu. Look for “Clone Formatting” and select it.

When you do, you’ll notice that the paintbrush icon on your toolbar is selected:

To apply the formatting you just cloned, select the cells you want to apply it to. Your cursor will change to a paint bucket. Click it over the selected cells and the conditional formatting will be applied to those cells. Now, test it. Add an “x” next to one of the other items to see if your conditional formatting is working:

Congratulations! You’ve successfully set up your first conditional formatting in LibreOffice Calc.

(NOTE: You can also apply the conditional formatting by indicating the cells to which the formatting should be applied in the Conditional Formatting window at the bottom. We indicated just A2 above, but you could include a range like A2:A6.)

Tutorial B – Conditional Formatting a Range with Colors

For this next tutorial, we’re going to get a little fancier. Nothing too crazy, but we are going to make the formatting conditional on the values in the cell and then apply background color to the cells. Our sample data are test scores.

First, let’s select the top test score in B2 and go to the “Format” menu, then select “Conditional Formatting” -> “Condition.”

We’ll get the same window as we got in Tutorial A. But things are going to be different from here on. We’re going to set up a color code that reflects test score ranges so we can quickly see which students are doing well and which may need extra help. To do this, in that window, leave “Cell value” as is and select the next dropdown. We’re looking for “is greater than or equal to”:

We want that option because the first condition we are going to enter is our top range of scores – 90 and up. Now, in the box next to that, enter 90. (NOTE: No need for quotes as this is a number, not text.)

Now, the style. We are going to apply a colored background based on these ranges. So, in the dropdown next to “Apply Style” select “New Style”:

Like we did above, we’re going to name our style, something like “grades_best”.

Next, click on the “background” tab then select “Color” and you’ll see this:

You can now pick the color you want to use as the background for that cell. I like green to mean that a student is doing well. So, I’m going with that:

When you’ve picked your color, select “OK.” That will return you to the conditions window. We have our first condition, but we wanted to assign colors based on ranges. That means we need additional conditions. In that window, click on “Add.”

When you click on it, you’ll get “Condition 2.” We’re going to follow what we just did for Condition 1, but we’re going to change a couple of things. First, in the dropdown next to “Cell value,” we don’t want “is greater than or equal to” anymore as our next range will have upper and lower values. So, we want “is between.”

Now, think carefully about how this will work. Our first Condition was equal to or greater than 90. We want this one to range from 70 to 89. Thus, will need to set the two values as 69 and 90 so it includes all the values we want but excludes 69 and 90. Once you have that, follow the same steps above to set the background color you want. I went with blue for doing okay and named that style “grades_good”:

We’re going to set up two more conditions. One is going to be from 60 to 69 and is going to be orange. The last is from 0 to 59 and is red! Like this:

When you have all your conditions in, click “OK.” The background color for B2 should now change to match the conditions you created. You can test it by changing the values in B2 to make sure everything works.

Now, as we did above, we can copy that conditional formatting and paste it to the other cells we want to have that formatting. Right-click B2 and select formatting then select the cells where you want to apply it and click with the paint bucket. All of those cells should now have the same conditions as B2.

There you go. That is how to use multiple conditions to format cells with a range of values.

Some Useful Notes

If you want to change the conditional formatting for a cell, be careful. If you right-click a cell with conditional formatting and select “Conditional Formatting” you’ll get this rather uninformative warning window:

In this version of LibreOffice, whether you select “Yes” or “No” doesn’t seem to matter, it wipes out the current formatting. So, don’t do that. Instead, select the cell you want to modify, then click on the “Format” menu at the top of the window and select “Conditional Formatting,” but don’t select “Condition”! Instead, click “Manage” and that will pull up the existing conditions. You can then modify the existing conditions from that window.

 789 total views,  8 views today

How To Fix PDF Forms Not Showing Contents in Okular (Linux)

Posted on by rcragun

I have had this problem happen twice now, and I don’t know exactly what is causing it, but I found a solution. Here’s the situation…

I received a PDF that has forms. Typically, that isn’t a problem as Okular is able to open most PDFs with forms and show both the forms and what has been entered into the forms. However, with this particular PDF, when I click on the “Show Forms” button at the top of Okular, I can see the information entered into the form like this:

(NOTE: I tried opening the PDF in Xournal and had the same problem.)

But when I click on “Hide Forms” (which is still showing on this PDF as “Show Forms” for some reason), all of the information entered into the form goes away, like this:

If I try to save the PDF as a different PDF in Okular or print it, nothing shows up. I found one website that had a suggestion for why this might be happening – the PDF may have been filled out in a web browser instead of using Adobe Acrobat Reader (or Okular) and, as a result, the PDF that was saved has some problem. I’m not sure what browser would be doing this, but that gave me an idea. If the problem is that it was filled out in a browser, perhaps I can open it in a browser then print it to PDF. So, that’s what I tried and it worked.

I opened the PDF in Brave (one of the browsers I have installed on Linux) and the PDF showed all the forms with all the information filled in:

I then printed the PDF (which flattens it and removes the forms) using the print button to a PDF:

The resulting PDF, while flattened, now showed all the information in the forms and allowed me to sign the document electronically.

I’m not sure how I solved this last time, but I figured I’d post this solution up here for the next time this happens so I can quickly take care of the problem and don’t have to try to figure out why form information isn’t showing up in my PDF on Linux using Okular.

 924 total views,  5 views today

LibreOffice – address label merge (from spreadsheet)

Posted on by rcragun

I’ve been using LibreOffice for over a decade and have only had to do an address label merge twice in that time – both of which have been in the last month. The first time, I actually did a form letter and a label merge; the second time I just had to do an address label merge. In the process, I realized that I totally forgot how to do it after the first time. Whenever that happens, I know that means I need to create a tutorial on here so I can remind myself how to do it if I ever need to do it again. So, here is my tutorial on how to merge labels in LibreOffice. (NOTE: I’m using LibreOffice 7.1.6.2.)

The idea behind a label merge, just like a form letter merge, is that you have a bunch of address information in a spreadsheet or database and, rather than having to enter all of that separately into a document to print labels, you’ll just have the software create the labels from the data you already have. LibreOffice is fully capable of doing this, but it’s not what I would call “easy” or straightforward. And, in fact, the first step will seem unrelated to merging labels, but it is necessary.

Part 1: Register Your Database with the Address Book Data Source Wizard

Before creating any labels, the first thing you need to do is create your database. I’m assuming you have a spreadsheet with addresses, like this one:

Make sure that you have column names at the top as those can become the field names in the database. It’s always a good idea to separate out your fields like I have done: first name, last name, street, city, state, zip. Of course, if you’re outside the US, your columns will be different. Adjust accordingly.

Okay, now we get to the weird part that is going to seem unrelated to merging these addresses into labels. A quick explanation may help. Basically, you need to turn this list of addresses into a database that LibreOffice has registered so it can then pull those fields when it generates the labels. This isn’t hard to do. LibreOffice will walk you right through it with a wizard. But it seems counterintuitive that you have to do this before you can start the mail merge. Just think of this as getting everything set up.

To create your database, click on File -> Wizards -> Address Data Source:

When you click on that, you’ll get this window:

This is Step 1 of 5 for the wizard. It’s basically saying, “Hey, let’s use existing data instead of having to enter new data.” Groovy. That will save us time. Unless you have your address data stored in Firefox or Thunderbird, select “Other external data source.” Then click on “Next” and you’ll get this next screen:

This is Step 2 of 5 for the wizard. On this screen, click on “Settings” and you’ll get a completely separate window:

(Did I mention that this is a bit confusing and complicated. It’s actually really smart, when you think about what the programmers did here, but it isn’t straightforward at all that you have to do all of this first. We’re only about halfway done.)

You have a lot of options you can choose from on this screen. I’m using a Spreadsheet, so I’ll leave that selected. But you can see from this screenshot that you choose from a variety of database sources:

Once you’ve selected the source for your data (again, I’m using a spreadsheet), select “Next” and you’ll get this screen:

Here, you need to choose your data source. Click on “Browse” and go find the spreadsheet you are going to use, like this:

Click on “Open” once you’ve selected it. That will take you back to the previous screen. If you want, you can click on “Test Connection.” Assuming everything worked, you should get this nice but relatively uninformative window:

Click on “Finish” and you’ll go back to the wizard:

This is Step 4 of 5 (note, Step 3 was skipped because I only had one field in my spreadsheet). On this screen, you can click on “Field Assignment,” which will let you match the fields in your imported data to the fields that are common for addresses. (NOTE: This is not required. You can skip this step if you want. It just changes the names of the fields later.) Here’s what the screen should look like:

Just to be clear, the Data source field is what LibreOffice is currently calling the newly imported data (in the image above, it just says “Addresses”). You’ll get a chance to change this later. The Table field (mine is called “Sheet1”) is the table in your spreadsheet where the address information was stored. You can then click on the various fields below where it says “Field Assignment” and align your information with the information options given, like this:

Once you’re done, hit OK and you’ll head back to the wizard.

This is Step 5 of 5. Phew. We’re almost done with this first part. You have a couple of options here. The first is a checkbox that says “Embed this address book definition into the current document.” You can unclick that if you want. Alternatively, you can leave it checked and make it part of the spreadsheet you are working on, which is fine. It really kind of depends on what your future plans are with this data. If this is data that you think you will use repeatedly for mail merges and you’ll probably be updating the data in the future, it might be a good idea to leave that checked. The second option is the one you definitely want to leave checked because we’re going to need it accessible for the label merge we’re going to do next. That option is also a checkbox that says “Make this address book available to all modules in LibreOffice.” Finally, you can name this address book. Call it something you’ll remember. I’m going with “folk artist addresses.” This changes the name that LibreOffice had temporarily assigned it (see the screenshot just above this). Once you’ve made your selections and named your Address Data, click “Finish”:

And once you hit “Finish,” nothing, right? Well, isn’t that weird.

Something did happen, it’s just “behind the scenes.” You have created a database that is now registered with LibreOffice. It’s just not showing it to you. If you want to see it, here’s how. Click on View -> Data Sources:

You’ll then see this:

You can see that I’ve got four Data Sources registered with LibreOffice. One of those is, of course, the one I just created: folk artist addresses. You can actually work with those data sources here if you’d like. Or, if you want to delete one (say, you made one by mistake or for a tutorial and you’ll never need it again), you can right-click on any of the databases and you’ll see this menu:

If you select “Edit Database File,” the LibreOffice database software will start up. If you select “Registered databases,” you’ll see a list of your databases with an option to select them and delete them, like this:

Okay. That does it for the prep work and part 1 of the tutorial. Now, with our address book registered, we can actually do the label merge.

Part 2. Label Merge In LibreOffice

Honestly, the first part is the hardest part. This next part goes pretty quick.

From any document in LibreOffice (it can be a Calc or Writer document; doesn’t matter), click on File -> New -> Labels:

You’ll then see this window:

Some explanation is in order. The big box is where you’re going to construct your addresses. I’ll walk you through that. The little check box at the top that has “Address” next to it is kind of nifty. If you’re a good LibreOffice user, when you first installed the software, you’ll click on Tools -> Options and fill in your User Data:

If you actually did that, you can click that little check box by “Address” and you’ll see your information populate the field:

This is a really slick way to, for instance, create business cards. But that’s not what we’re trying to do. So, uncheck that box and let’s get to merging some labels. First, in the drop down menu under “Database,” select the database we just created (and now Part 1 makes sense):

In the drop down menu under “Table,” select “Sheet1” (it’s the only sheet we had; we could always have named it something else). So, your window should look like this now:

Next, we start working with the drop down menu under “Database field.” In that menu, you should see all the fields you had in your spreadsheet:

Let’s build our address label. Select “firstname” then click on the little arrow pointing left:

That will insert that field into the label box, so you’ll see this:

Now, build the rest of your label, keeping in mind that you’ll want to add spaces and punctuation in between the fields, so it looks like this:

Once you’ve got it set up how you want it, we have a few other things we need to do. At the bottom, you need to choose between “Continuous” or “Sheet.” I’m assuming the Continuous is for printers that can print on labels that are connected. I typically print on individual sheets, so I select that. On the bottom right, find the “Brand” you want and then the “Type.” (Honest Aside: LibreOffice doesn’t always have every option I need for this, but you can usually find something that will work.)

We’re almost done. At the top of that window you’ll see two other tabs: “Format” and “Options.” If you want to customize the dimensions of your label, click on the Format tab. I rarely use this option, but it might be useful if you have a custom label you want to print. There is one thing on the “Option” tab that you probably want to select: “Synchronize contents”:

This will allow you to synchronize any formatting you do to the labels later. Trust me, you’ll want this option like 99% of the time. Once you’re done and have your label selected, click on “New Document” at the bottom.

You’ll then get an entirely new Writer file with labels spread out across the document and your fields highlighted in gray:

What you won’t see, unfortunately, is the actual data in those fields. That will come later. What you’ll want to do now is customize your label formatting. So, if you want to change the font, the font size, make the text bold or italicized, now is the time to do it. And, do it all in the top-left cell. In my document, I increased the font and made everything bold:

Remember when I said to choose “Synchronize contents.” Yeah, now is when you’ll want that. You don’t have to format every individual label. Just format the first one, as I said. And when you’re ready, click on the “Synchronize Labels” button in the floating window that popped up:

When you do, the formatting from the first cell transfers to all the other labels. Way easier than formatting each cell individually.

Now, to print the labels. This part is also, unfortunately, a little confusing. Click File -> Print or just hit the printer icon on the toolbar and you’ll get this message:

“Your document contains address database fields. Do you want to print a form letter?”

What an unfortunate prompt. We’re doing a label merge, so “No.” But don’t choose No! We may not want to print a form letter, but there isn’t a separate prompt for doing a label merge. So choose “Yes.” (Nudge to LibreOffice programmers – fix this.)

When you do, you’ll get this window:

This is a little confusing, but it’s basically that same Data Source window you saw in Part 1, but now we are applying it. You’ll see your Address Data Source on the top left. If you select your sheet (Sheet1 in my screenshot) you’ll see all your addresses. You have a couple of options here. You can choose not to print all of the addresses. This is in the Records option on the bottom left. If you want to test this with a single sheet, for instance, you can choose just the first XX addresses and test this out. On the bottom right, you can choose to save this as a File or send it directly to the Printer. Up to you. If you’re wary and don’t want to waste label sheets, this might be a good option. Or if you need to print these on a different printer, that would be helpful. But if you’re ready to print, select “OK” and you’ll get one more chance to change your mind because you’re finally going to see THE LABEL MERGE! (Woot! Rejoice!):

The screenshot above shows you the final window I get before I actually print the labels. And, assuming everything worked correctly, you’ll finally see the address information merged into the labels. Success!

This is not a simple process but there is, as with all things LibreOffice, a lot of customizability, which is why I prefer LO over Microsoft products.

Anyway, that’s how you do labels merge in LibreOffice. And, bonus, if you ever need to use that same address information in the future, it’s already registered in LibreOffice so you can just do this again. If not, you can delete it as I illustrated above.

 5,634 total views,  15 views today

R – Finding and Setting Working Directory

Posted on by rcragun

R, like many software programs, likes to have a folder or directory on your computer to operate in. When you start R, depending on how you start it and whether or not you previously saved your session, it’s likely that R will set a default “working directory” or folder where it is going to look for files and folders and where it will default to storing whatever it is you are working (e.g., script files, etc.). Many R beginners don’t know that you can change the working directory AND it is really useful to do so.

There are two ways to change your working directory if you are using R with RStudio – from the console or using the menus in RStudio. I’ll show them both.

To begin with, you can use the following code to get your current working directory:

getwd()

When you enter this into the console or run this from a script file, R will indicate what the current working directory is, like this:

In the screenshot above, you can see that I had set my working directory to a folder called “new analysis” for a project I am working on, but it also shows the entire folder hierarchy to my working directory: “/home/ryan/Dropbox/Ryan/writing/transing god/new analysis”. Knowing what my working directory is, makes it much easier to change to my working directory.

To change your working directory, you can copy the folder hierarchy that R provided and simply change it to the new location. So, for instance, if I wanted to change my working directory to a different project that is stored in a different folder, I can just change the folder hierarchy with the following command:

setwd("/home/ryan/Dropbox/Ryan/writing/futurism studies/data/")

Basically, getting your working directory uses the code “getwd()” with nothing in the parentheses, and setting your working directory uses the code “setwd(“/path/to/folder/”)” with the folder hierarchy included inside quotes inside the parentheses. This is how you get and set your working directory from the console.

If you’re using RStudio to interface with R, you can also set your working directory using the point-and-click menus. Click on Session -> Set Working Directory -> Choose Directory:

Once you click on “Choose Directory,” a window will pop up that will let you select the folder that you want to be your working directory:

Navigate to the folder you want to use as your working directory, select it, and hit “Open” and that will set your working directory. You should also see the corresponding code in the console that matches what we did above.

You can, of course, check to make sure that your working directory has changed by using the code above – “getwd()”. It should now be your new working directory.

Why might you want to change your working directory? The major advantage to doing this is that you no longer have to give the full path length to files you want to import, export, or work on. You can simply give the relative path based on the working directory. For instance, if I want to save a table of data from my environment, if I have set my working directory, I don’t have to specify the entire path to the file when I save it, I can simply save it to my working directory with the name, like this:

write.csv(MyData, file = "MyData.csv")

The code above will save the environment object “MyData” as a CSV file into my working directory and call it “MyData.csv”. If I haven’t set my working directory and want to save the same object, I would have to specify the directory where I want to save it, like this:

write.csv(MyData, file = "/home/ryan/Dropbox/writing/someproject/MyData.csv")

Setting your working directory also makes it easier to import files as you only have to use the name of the file you want to import and not the entire folder hierarchy to the file. With my working directory, I could import a CSV file like this:

dataframe1 <- read.csv("MyData.csv", header=TRUE)

Contrast that code with this code that would be required if I had not set my working directory:

dataframe1 <- read.csv("/home/ryan/Dropbox/writing/someproject/MyData.csv", header=TRUE)

Using a working directory saves time and makes it much easier to import and export files from R.

BONUS:

If you ever need to find your folder hierarchy, this is relatively easy to do on Linux or Windows (no promises on Mac). On Linux, just open a file manager (like Dolphin), navigate to the folder you want, then select the folder hierarchy in your file manager, like this:

You can simply copy and paste that into R. If you want to use the console to do it, you can also navigate to the folder in a console on Linux then use the “pwd” (path to working directory) command to get your current directory:

 1,254 total views,  3 views today

Launching HandBrake 1.4.0 in Kubuntu 21.04

Posted on by rcragun

The fine folks at HandBrake have updated their software and distribution system to a Flatpack approach. Their Flatpack for Linux is based on Gnome, not KDE, which is fine, but it does mean that the GUI is no longer skinned with my system settings on KDE. Oh well…

The bigger issue is that, with a flatpack install, HandBrake is no longer registered with the KDE system and has to be launched from the command line. That isn’t particularly onerous, but it is annoying when I use KDE’s KRunner to launch most of my software (Alt + F2). There is a solution to this that works just fine: add HandBrake to the Applications in the Application launcher.

To do this, right-click on the Application launcher and select “Edit Applications”:

In the window that comes up, select “New Item” to add HandBrake as an Application:

You’ll get a Window to name the new item:

You can then fill in the following details in the General tab:
Name: HandBrake
Description: HandBrake
Comment: launch Handbrake from Flatpack
Command: flatpack run fr.handbrake.ghb

If you want, you can also download the Handbrake logo and select it by clicking on the little square next to Name and Description:

When you’re done, hit “Save” and HandBrake will be registered with the system as an Application.

Now, when you use KRunner (Alt + F2), HandBrake will pop up as an option:

 1,307 total views,  2 views today

Linux FFMPEG – Flip Vertical Video to Horizontal

Posted on by rcragun

I was recently shooting some videos on my phone to share with someone else and accidentally flipped the phone before filming a clip. I never film vertically as I know that our playback devices are not designed to playback vertical footage. But I screwed up and got a clip that was vertical. Since I would consider sharing such a clip “bad form,” I needed a quick and easy way to flip the clip. Enter FFMPEG.

This can be done very easily with a single line of code at the console or terminal in Linux, assuming you have FFMPEG installed. Here’s the line of code:

ffmpeg -i input_video.mp4 -vf "transpose=2" output_video.mp4

Here’s what the code does. First, it calls “ffmpeg.” Then you tell it what video to input “-i input_video.mp4.” (The “-i” tells FFMPEG that what follows is the input.) If you’re not in the same directory as the video, you can simply add that information (e.g., /home/user/input_video.mp4). Next, the “-vf “transpose=2″” tells FFMPEG to first “-vf” create a filtergraph (basically, apply a filter) which is “transpose.” The “2” indicates how much to rotate the video – 90 degrees in this case. Finally, pick a name for your file and a file extension. That’s it. That will rotate your video from vertical to horizontal.

Here’s an actual example of code that I used:

ffmpeg -i /home/ryan/Desktop/vertical.mp4 -vf "transpose=2" /home/ryan/Desktop/horizontal.mp4

Sources for this post. FFMPEG’s documentation (here and here) and this post.

 1,429 total views,  1 views today

Linux – Batch Convert .wav to .flac

Posted on by rcragun

I ran into a weird issue the other day where SoundConverter, a GUI for converting audio files, was generating flac files that my audio player couldn’t see. I’m still not exactly sure what the problem was, but in trying to solve the problem, I went ahead and wrote a command to batch convert a folder of .wav (WAV) files to .flac (FLAC) files using FFMPEG. I figured I’d put it up here for me in the future and in case anyone else finds it useful.

First, navigate in a terminal/console to the folder where the audio files are you want to convert to flac. Then run the following command:

for i in *.wav; do ffmpeg -i "$i" -c:a flac "${i%.*}.flac"; done

Breaking this code down… The first part “for i in *.wav” starts the loop by telling the computer to loop through every file in that folder. The second part tells the computer what to do (“do”) with each of those files: load the ffmpeg software and for each file “$i” convert it to flac “-c:a flac”, renaming the file with the same name as before but with the flac file extension “”${i%.*}.flac””. (See here for what these characters do.) When that is complete, the loop is done.

 4,429 total views,  7 views today

LibreOffice – KDE Integration Package for Skinning/Color Management

Posted on by rcragun

Super simple note for future reference and for anyone else running into this issue. I upgraded my version of LibreOffice in Kubuntu 20.10 to a pre-release version to fix a bug. I was running version 7.1.4.2.

When I purged the version of LibreOffice that shipped with Kubuntu 20.10 (6.X – I don’t remember exactly which version), that also purged the KDE integration package that helps LibreOffice interface with the window manager and makes skinning LibreOffice much easier. As a result, when I would change my Global theme in System Settings, only part of the LibreOffice windows would change to reflect that theme. In particular, the sidebar on the right side of LibreOffice wasn’t changing colors with the rest of the window. This was making it impossible for me to see the names of different styles and also looked really weird.

The solution was easy. In Synaptic, search for the “libreoffice-kde” integration package and install it. Now, when I change my Global Theme, the LibreOffice windows change to reflect that.

In short, if you purge the version of LibreOffice in the repositories in KDE and install a newer version, make sure you also install the libreoffice-kde integration package or your LibreOffice windows will behave strangely.

 1,380 total views,  1 views today

Linux Server – tar files and directories

Posted on by rcragun

I recently had to move all my websites from one virtual private server (VPS) to another. When I only had a few such websites, I was okay with using SFTP (via Filezilla) to download all of the files and then upload them to the VPS. It took a while but I was okay with that. With about a dozen websites I host on my VPS, that was just not an option. It was time to finally try to figure out how to use tar more effectively on my server.

Why tar? Tar is similar to zip in that it combines lots of files and/or directories into a single file, a tarball or archive, which makes it much easier and way faster to move. Computers can move one large file faster than they can move lots of small files that, combined, make up the same size. When you have to move roughly 8 gigabytes and hundreds of thousands of files, it is far easier to do so by putting all of those files into a single tarball than moving the files individually. That’s why tar is way better for what I was trying to accomplish.

Sidenote: Tar is just a format for packaging all of the files/directories together. Typically, the files are also compressed using something like gzip, leading to a tarball with the extension “tar.gz.” It is possible to just combine the files without compressing them as will be detailed below.

Why was I so reticent to use tar to move my files? Because my prior experience with tar had resulted in several tarbombs, which are a nightmare. Basically, I had unpacked a tarball into the wrong directory, which resulted in thousands of files being in the wrong place, necessitating me having to figure out which files I should keep and which I should get rid of individually. That took more time than I saved by using tar. And since I was doing everything via SSH on a remote VPS, there was no easy way to clean up the mess. Even so, it was time to use tar. So, I bit the bullet and figured out how to do this. This post is my guide on how to carefully use tar but avoid tarbombs, which suck. Note, GNU has an entire manual on tar that goes into much greater detail than my post.

What do all those letters mean?

Since we’re using a console or terminal to package all these files and not a GUI, we have to specify what we want to do with the files using some letters. That’s what the letters do. Not to worry, though – I’m not going to go through all of them. There are literally dozens of options. I’ll keep this simple. The basic structure of the tar command is as follows:

tar [letters to specify what to do] [output tarball] [files or directories to add]

I’ll give some specific examples of tar commands below, but, first, let’s cover what the letters we’ll be using mean.

  • c – “create”: This tells tar to create an archive, in contrast to modifying a tarball. (Note: This letter alone cannot create an archive.)
  • f – “file”: This tells tar to modify files relative to the archive. (Note: Just c and f can work to create a tarball as will be shown below.)
  • v – “verbose”: This tells tar to show the progress of the archiving process by showing which files have been added to the archive.
  • z – “gzip”: This tells tar to compress the files in the tarball using gzip.
  • j – “bzip2”: This tells tar to compress the files in the tarball using bzip2.
  • x – “extract”: This tells tar to extract the files in a tarball.

A couple of important notes here regarding the letters. First, the order of the letters does not matter. Second, letters that do the same things should not be included in the command (e.g., “z” and “j” should not both be used).

Basic Examples:

Creating Tarballs

I put a couple of my papers and some images into a folder to use to demonstrate basic uses of tar. Here’s a screenshot so you can see what we’re working with:

First up, I’ll create a tarball of the entire test folder but with no compression and use that to show a couple of important elements of the process. Here’s the code:

cd /home/ryan/Desktop
tar cf test.tar tar.test.directory/

The code above changes my directory (“cd”) to the parent folder (Desktop). Then it calls the “tar” software and tells it to create an archive called “test.tar” and file inside it (“cf”) the entire test directory “tar.test.directory/”. What this code doesn’t do is compress the files. This can be seen when comparing the size of the folder – 5.4 mb – with the size of the newly created tarball – 5.4 mb.

Quickly, try switching the order of the letters, from “cf” to “fc” and you’ll see that the outcome is the same. Also, if that is all you switch in the code, you’ll notice if you re-run the same command that tar will not warn you that it is going to overwrite the previous tarball, it simply does it.

One more item to note that is actually really important, particularly when thinking about extracting an archive, is the folder structure inside the archive. If I open the archive using Ark, you’ll see that, because I navigated to the parent directory in my terminal before creating the tarball, the folder structure inside the archive is from the folder where I created the tarball (in this case, the Desktop directory).

I’m going to create the same tarball but I’m not going to navigate to the parent folder and instead will tell tar which folder to compress and where to store the new tarball:

tar cf /home/ryan/Desktop/test.tar /home/ryan/Desktop/tar.test.directory/

Functionally, this is the exact same command and archives all the same content. However, look at the directory structure inside the tarball when I open it with Ark:

That tar creates a different directory structure depending on the code you use is important, particularly if you want to avoid tarbombing. Why is this important? When you extract the tarball, the same directory structure that is in the archive will be created. If you don’t know what the directory structure is inside the tarball and extract it, that can result in all sorts of problems, some of which I’ll address at the end of this post.

The lesson here: the folder structure is based on what you enter into the command to create the tarball. So, if you don’t want lots of folders in your tarball, navigate to the folder above it and create it there. Otherwise, you’ll get the same folder structure.

Let’s add two additional letters: “v” for verbose so we can see what is added to the tarball and “z” to compress the files that are being added to the tarball. Here’s the code (remember, the order of the letters doesn’t matter):

tar cfvz test.compressed.tar.gz tar.test.directory/

This command calls the tar software, tells it to create (“c”) a tarball and add files (“f”), show the progress (“v”) and compress the files (“z”), saving the resulting tarball as “test.compressed.tar.gz” and the last piece is what should be put into the tarball. Note the modification to the file name to indicate that the tarball has been compressed – “.gz”. This extension is usually a reflection of the compression format. So, if you go with bzip2, it would be “.bz2” instead of “.gz”. Here’s how it looks in the terminal.

The resulting tarball is only 3.4 mb, illustrating that the contents were compressed as they were added to the archive.

Variations on the above command might include replacing the “z” with “j” to compress using bzip2 or “J” to compress using “xz.” Additionally, appending a “p” to the letters will preserve the permissions of the files and directories that are added to the tarball (though that is done by default, so it isn’t necessary to include it).

Modifying a Tarball

After creating a tarball, it’s possible you may need to change the tarball by either adding files to it or deleting files inside it. Here’s how to do each of those.

To add a file to a tarball, use “r” and, of course, “f”, like this:

tar rf test.tar file-to-add.odt

This command calls the tar software and tells it to append (“r”) a file to the archive (the “f” is necessary to tell the software to make changes). The tarball that is modified is next “test.tar.” And what is to be added is last “file-to-add.odt.”

Assuming the tarball you want to add a file to is compressed, files cannot be added. Attempting this will likely give you the error, “tar: Cannot update compressed archives.” Instead, you would need to extract the archive, make whatever changes you want, then create a new compressed tarball.

It is also possible to delete a file from a tarball. This doesn’t involve a letter but a word, “–delete.” The structure of the command is a bit different. Here’s what your code might look like: 

tar --delete --file=test.tar file-to-delete.odt

This command calls the tar software and tells it you want to delete a file “–delete.” You then have to tell it which tarball and which file, which is done with the “–file=” option, which specifies the tarball and then the file is added at the end.

If you aren’t sure what files you have in your tarball, you can always list those using the “–list” command:

tar --list --file=test.tar

This is particularly helpful if you are looking for a specific file to remove from a tarball as it will also tell you if the file is in a subfolder inside the archive. If so, you would need to modify the code to take that into account:

tar --delete --file=test.tar "folder1/folder2/folder with space/file-to-delete.odt"

Do note that, just like adding files to a compressed tarball, deleting files from a compressed archive isn’t possible.

Extracting a Tarball

Here’s how to extract the files inside a tarball. The basic structure is the same, though, there are some things to consider. First, to extract a tarball, replace the “c” from above when creating it with an “x” which means “extract.” So, with our uncompressed tarball, we would extract it by using the following command:

tar xf test.tar

This will call the tar software, the “x” tells it to “extract” the files (“f”) and “test.tar” is the name of the tarball that is being extracted. Note that this code doesn’t specify where to extract the tarball. That can be added as an option that requires adding “-C”. If it’s left blank, the tarball will be extracted in whichever directory the console/terminal is in. Extracting to a specific directory would like like this:

tar xf test.tar -C /extract/into/this/directory

Of course, if your tarball is compressed and/or you want to see the progress of extracting the tarball, add the corresponding letters. If the tarball is a “tar.gz” you’ll need to add the “z” to decompress the files and “v” to see the progress, like this:

tar xfzv test.tar.gz

Lastly, keep in mind that the folder structure inside the tarball will be replicated when the files are extracted. What does that mean? If you extract a tarball into a folder called “home” but inside the tarball the files you want to extract are stored inside nested folders like “home/ryan/Desktop/archive”, when you extract the tarball, the files will end up in “home/home/ryan/Desktop/archive.” See below for what to do in such a situation.

Rules for Using tar:

Rule #1: Pay attention to the folder or directory structure when creating and extracting a tarball.

Rule #2: You cannot add or delete files inside a compressed tarball, only in an uncompressed tarball.

Bonus:

Let me give a specific situation that will illustrate this for a Linux server (I may be speaking from experience here). Imagine you just downloaded the latest version of WordPress and want to extract it into the following directory: /var/www/example.com/public/. First, check the directory structure of the tarball. In this case, the fine folks who package WordPress archive the files inside a folder called “wordpress.” As a result, when you extract the tarball, it is going to create a folder inside the “public” folder called “wordpress” when, in fact, you want the files to be directly stored into the folder “public.” That’s a problem. How do you move those files? The Linux move command, “mv” can do it, but it’s kind of tricky how:

mv /var/www/example.com/public/wordpress/* .

This tells the OS to move all the files in the wordpress directory (the asterisk “*” does this) up one level (the period “.”). Once you do this, you should then remove the wordpress directory:

rm -R wordpress/

 1,054 total views,  1 views today