R – find cases (rows) that match specific criteria

I regularly need to find a specific case or set of cases that meet some criteria when analyzing data, often so I can modify those values for one reason or another. The easiest way I have found to find such values in R is the “which” function.

As with most of my R examples, I’m going to use the 2010 wave of the General Social Survey (R version here) to illustrate. You can open that file in R and follow along.

In the 2010 GSS there is a variable for race (RACE). The options are: 1 = WHITE, 2 = BLACK, 3 = OTHER. To find all of the cases with a “3” in the dataset, I would use the following code:

which(GSS2010$RACE == "3", arr.ind = TRUE)

Here’s what the command is doing…

which” is the function that tells R to search for information that meets the criteria detailed in the parentheses.

GSS2010 is the name of the dataset.

RACE is the name of the variable in the dataset. By including the name of the variable, we restrict R to searching just inside that variable rather than the whole dataset. (The $ tells R that RACE is a variable inside the GSS2010 dataset.

The “==” indicates “equals” in R.

The target value, which can be text, characters, or numbers, goes inside the quotes. In this case, we wanted to find all of the cases with the number “3” which is code for “OTHER.”

arr.ind = TRUE tells R to include the index if the result is an array.

In the 2010 GSS, if you type in the code above, you’ll get a list like this:

 [1]    1   12   14   19   27   28   42   44   46   50   64   73   96   97  101  102  119  120  121  123  124  130  133  140  145  147  151  152
 [29]  153  154  159  161  180  185  190  194  195  199  211  213  217  220  230  245  263  275  278  287  288  295  297  301  305  312  314  318
 [57]  333  339  345  348  349  371  373  381  403  420  441  446  458  464  465  473  475  477  478  479  483  489  495  501  505  507  508  520
 [85]  550  554  561  564  567  591  593  631  704  712  713  715  716  732  741  749  770  776  792  793  805  807  823  824  829  877  901  956
[113]  973 1092 1112 1125 1186 1193 1218 1224 1264 1281 1291 1304 1307 1331 1336 1342 1345 1347 1352 1355 1356 1364 1365 1411 1423 1440 1441 1442
[141] 1444 1445 1446 1449 1451 1513 1523 1526 1528 1532 1534 1547 1550 1552 1556 1557 1559 1562 1564 1567 1568 1569 1570 1571 1572 1573 1574 1575
[169] 1576 1656 1660 1735 1764 1913 1933 1935 1993 2010 2011 2018 2019 2022 2038

The [1] is indicating that this is the first response. The [29] indicates that the next number is the 29th response. The numbers after the brackets (“[ ]”) indicate the row where that response was found. Thus, I know that the 161st row in my dataset has the value 3 in the variable RACE.

We can check this by using the following code:

GSS2010[161,"RACE"]

The result should be:

[1] 3

BONUS:

Should you want to modify the value for an individual observation, like the one we just examined, you could use the following code:

GSS2010[161,"RACE"] <- 2

This would change the values for that case from “3” (OTHER) to “2” (BLACK). I’m not really sure why you would want to do this in this instance, but, now you can. (There is a scenario when you might, but there are better ways to recode data.) Basically, the “<-” tells R to set the value of that specific observation to 2, overwriting the 3 that was there.

And if you wanted to change all of the values from 3 to 2, since you have a massive list, the easiest way would be to save all those values as a list, then have R change all the values in one fell swoop, like this:

OTHERRACELIST <- which(GSS2010$RACE == "3", arr.ind = TRUE)
GSS2010[c(OTHERRACELIST), "RACE"] <- 2

The above two commands would create a list in your environment called “OTHERRACELIST” that includes all of the row numbers of the cases with a 3. The second command then tells R to look inside the GSS2010 dataset and use the list (c(OTHERRACELIST)) to find all the rows you want changed in the RACE variable to “2.” That will then change the code for all of the people coded as “3” into a “2.”

A script file with the above commands is available here.

R – delete one or several variables in a dataset

I regularly create variables while analyzing data and then find that I need to delete a variable I created. At times, I just want to get rid of a variable in a dataset (’cause screw that variable). This short tutorial will explain how to delete a variable (or multiple variables if needed).

As with most of my R examples, I’m going to use the 2010 wave of the General Social Survey (R version here) to illustrate. You can open that file in R and follow along.

To completely remove a variable from a dataframe, you need to tell R to copy the dataframe minus the variable you want to delete. Here’s the code:

GSS2010 <- subset(GSS2010, select = -(OCC))

Here is what the code above does…

GSS2010 is the name of the dataset. Typically, when I use the subset function, I do so to create a different dataset. However, in this case I actually want to overwrite the dataset, so I’m actually naming the new dataset the same thing as the old dataset, which, effectively, overwrites the dataset, getting rid of the unwanted variables in the process.

The “subset” function tells R that you want to take part of an existing dataset. It’s a very useful function for selecting, for instance, all the men in a sample or all of the people who live in a certain region.

After the “subset” function, inside the parentheses, is the name of the dataset from which we are taking a subset, GSS2010. After the comma inside the parentheses is code to tell R how to select the subset. In this case, we use the “select =” command to tell R that we want it to select a specific variable. However, the “” before “(OCC)” actually tells R to select all the other variables BUT not the OCC variable for the subset. Thus, “-(OCC)” tells R to select the entire dataframe except the variable OCC for the subset. In effect, OCC is deleted but, to get there, you actually have to tell R to keep everything but that (pretty stupid, honestly).

NOTE: This is an instance in R when you don’t need to put the name of the variable in quotes (e.g., (“OCC”)) nor do you need to indicate which dataset the variable is in (e.g., (GSS2010$OCC)) since the dataset is already referenced in the subset command.

To remove multiple variables at the same time, the above command can be modified slightly to include other variables by putting them into a vector:

GSS2010 <- subset(GSS2010, select = -c(YEAR, WRKSTAT))

By changing what comes after the “select =” component in the parentheses to a vector (c indicates a vector in R), you can indicate multiple variables that you want deleted from the dataset in one command. Thus, in the above code, the variables YEAR and WRKSTAT would both be deleted from the dataset.

Because it is R, there is always another way. Here are two alternative lines of code that will do the same thing, the first is for removing a single variable and the second is for removing multiple variables:

GSS2010 <- GSS2010[,-match(c("EVWORK"), names(GSS2010))]
GSS2010 <- GSS2010[,-match(c("EVWORK", "PRESTIGE"), names(GSS2010))]

The logic in the above code is very similar, using the “match” command instead of subset.

You may find some tutorials that suggest you can remove a variable from a dataframe/dataset using the following code:

GSS2010$GOD <- NULL

What this command does is actually remove all of the data in the variable GOD. However, the variable remains in the dataset, it’s just empty. I prefer one of the above approaches because they completely remove the variable from the dataset.

Here’s a script file for these commands.

R – create variable filled with zeros

I ran into a situation where I needed to add a variable to a dataset. I knew that I was then going to modify some of the values in the variable, but most of the values were going to be zeros. So, I wanted to create a new variable and fill it with all zeros.

As with most of my R examples, I’m going to use the 2010 wave of the General Social Survey (R version here) to illustrate. You can open that file in R and follow along.

Here’s the code I used to create the variable:

GSS2010$TEMPANALYSIS <- replicate(2044, 0)

Here is what the code above does…

GSS2010 is the name of the dataset into which I wanted to create the variable. In this case, it is a copy of the 2010 wave of the GSS.

TEMPANALYSIS is what I called the variable. (The “$” tells R that it is a variable in the dataset.)

The “replicate” function tells R to replicate the second value in the parentheses (0) the number of times noted as the first value in the parentheses (2044). I used 2,044 because that is how many cases there are in the dataset. You can obviously adjust the value for the number of cases in your dataset/dataframe. If you have 320 cases, adjust it to 320.

If you don’t include the exact number of cases, you’ll get an error like this:

Error in `$<-.data.frame`(`*tmp*`, TEMPANALYSIS, value = c(0, 0, 0, 0,  : replacement has 2042 rows, data has 2044

That error is saying that you tried to add a variable but R needs to know what to put in every one of the rows and since it is short 2 rows, it can’t do it.

Of course, with R, there is always another way to do something. Here’s an alternative command that will do the same thing:

GSS2010$TEMPANALYSIS2 <- rep(0, times=2044)

I won’t repeat the description of the dataset and variable but will detail what the rest of the code is doing.

rep” tells R to repeat the first value in the parentheses (0) the number of times specified as the second number in the parentheses (2044; technically, the “times=” portion is not required.

Here’s a script file for these commands.

Unethical Amazon Review Modifications

I don’t always review products on Amazon. I don’t have the time. But there have been two instances over the past year when I have been contacted by someone because of a review I wrote on Amazon. Both times, these individuals have tried to bribe me to remove my negative review of the product. Here’s the latest email exchange over a backup cellphone charger that was a piece of garbage:

First Email:

taylor jack taylor.jack0528@outlook.com Tue, Dec 17, 2019 at 10:30 PM
To: “ryantcragun@gmail.com” ryantcragun@gmail.com

Hello,rcragun.
I’m Anna.I am a real person.Since everybody’s time is very precious,I just go directly to the topic.
Here is your review.
https://www.amazon.com/gp/customer-reviews/R1T2C38HIJ7JJQ/ref=cm_cr_getr_d_rvw_ttl?ie=UTF8&ASIN=B07SWTGDVW
I am very sorry to hear about the issues you’ve had with your item.Are you willing to help me to delete your review? If you have deleted your review, please tell me,We will give you an Amazon gift card.
Looking forward to your good news and reply !

My response:

From: Ryan Cragun ryantcragun@gmail.com
To: taylor jack taylor.jack0528@outlook.com
Subject: Re: Amazon compensation

I consider your proposal completely unethical.
You should be ashamed of yourself.
Best,
Ryan T. Cragun

I stopped responding after this, but have since received three more emails:

Email Two:

From: “taylor.jack0528” taylor.jack0528@outlook.com Using MailMasterPC/4.13.2.1001 (Windows 7)
To: “ryantcragun@gmail.com” ryantcragun@gmail.com
Subject: Re: Amazon compensation

From my perspective, we are very eager to compensate those customers who trust us but are hurt by us.
So please give us a chance. We will offer a $ 30 Amazon Gift Card.
Hope you understand us, our life and work are not easy.

Email Three:

From: “taylor.jack0528” taylor.jack0528@outlook.com Using MailMasterPC/4.13.2.1001 (Windows 7)
To: “ryantcragun@gmail.com” ryantcragun@gmail.com
Subject: Re: Amazon compensation

Time is limited and tight!!
How is thing going?
If you have deleted your review ,please tell me, I will give you $30 right away.
This is your review.
https://www.amazon.com/gp/customer-reviews/R1T2C38HIJ7JJQ/ref=cm_cr_getr_d_rvw_ttl?ie=UTF8&ASIN=B07SWTGDVW
Thank you.

Email Four:

From: Bessierh5 Nielsenqvl99 trisnatudd3cx@gmail.com
To: ryantcragun@gmail.com
Subject: $30 Amazon gift card

Hello rcragun.This is the final mail to you.
This is your review.
https://www.amazon.com/gp/customer-reviews/R1T2C38HIJ7JJQ/ref=cm_cr_getr_d_rvw_ttl?ie=UTF8&ASIN=B07SWTGDVW
If you are willing to delete your review ,I will offer an Amazon gift card worth $30.
If you have deleted it ,please tell me, I will give you $30 right away.
We will thank you profusely and even find a homeless kitten to hug on your behalf. We will waiting for you.
Again, this is the last email you will receive from us so we really do hope you are enjoying your purchase.My most sincere regards, I hope that you can reply to me as soon as possible.
Thank you and please have a fantastically glorious day.
Sincerely
Anna

With the previous company, I told them I wouldn’t delete my review. They were okay with that. They said that they would send me a new pair of headphones (the other ones died within a year) to make things right and hoped I would update my review to reflect that. That seemed fair to me and that’s what I did. They were okay with me leaving my review but wanted me to note that they tried to make things right.

That’s very different from what this person is doing. They are basically trying to bribe me to remove negative information from Amazon’s website so people will be misled about the quality of their product.

The product is NEXGADGET’s Solar Charger Power Bank. As I noted in my review, it was basically a useless brick during a 5-day hike in Wyoming. It didn’t keep its charge for a single day and wouldn’t charge in sunlight. Maybe I got a bad item. But that still speaks to the production quality and my review should stay on the website. NEXGADGET shouldn’t be trying to hide negative reviews but rather trying to make a better quality product.

Update 1/8/20:

I sent a response to their last email hoping to get more information.

My 2nd Response:

Created at: Tue, Jan 7, 2020 at 9:02 AM (Delivered after 0 seconds)
From: Ryan Cragun ryantcragun@gmail.com
To: Bessierh5 Nielsenqvl99 trisnatudd3cx@gmail.com
Subject: Re: $30 Amazon gift card

Hi Anna,
I’d like to know your real name and who you work for. Please provide that information and maybe I’ll consider your request.
Best,
Ryan

Here’s their response:

Email Five:

Created at: Wed, Jan 8, 2020 at 3:32 AM (Delivered after -325 seconds)
From: Bessierh5 Nielsenqvl99 trisnatudd3cx@gmail.com
To: Ryan Cragun ryantcragun@gmail.com
Subject: Re: $30 Amazon gift card

I understand your mood,but
On the one hand , this is not a bribe,this is just a compensation,we just hope that you are satisfied with this transaction.
On the other hand , this is not a violation ,because this shopping platform has this option after all,and the choices are in your hand.I just need you to delete it ,not a good review.
So what do you think of it ?
please give me a chance.
If you have deleted it ,please let me know, thank you!

And my final response:

3rd Response

Wed, Jan 8, 2020 at 6:41 AM (Delivered after 0 seconds)
From: Ryan Cragun ryantcragun@gmail.com
To: Bessierh5 Nielsenqvl99 trisnatudd3cx@gmail.com

Definition of BRIBE: “persuade (someone) to act in one’s favor, typically illegally or dishonestly, by a gift of money or other inducement.” You are trying to persuade me to act dishonestly with a gift of money. It is literally the definition of a bribe. Please stop emailing me or I will report you to Amazon.com directly.

To be fair, this is probably some underpaid individual in China who sees this as an opportunity to make enough money to survive. It’s just a job to someone. And my ethical pleadings will never persuade them to stop doing what they are doing because they need to eat. I get that. But it’s still a bribe.

Change Doorbell Sound on Ring App and Amazon Echo

I’ve had a Ring doorbell (and security system) for quite a while. I never bought the chime that goes with the doorbell because it has always worked through my Amazon Echo devices. However, I only recently learned that you can change the notification sounds you get when someone rings your doorbell. However, how you do this requires clicking through almost a dozen screens in the Ring app and I can never remember it. So, here’s how to do it.

Change Doorbell Sound on Ring App

I’ll start with the notification sound you get on your phone through the Ring app when someone pushes the doorbell. First, open the Ring app and you’ll be on the Dashboard or Home screen:

The dashboard or home screen.

Click the three lines in the upper left corner to open that menu:

Menu options.

Select “Devices”:

The list of devices.

Now select your Video Doorbell (mine is called “Front Door”):

Options for your Video Doorbell.

You’ll have to scroll down (at least, I did), to see the settings icon (the gear). Click on that:

These are the settings for your video doorbell.

The settings you want are the Alert Settings. So, click on that:

The alert settings for your video doorbell.

The second option down in the screenshot above is for the chimes that you can use if you have a separate chime for your device. I don’t. So, what I want to change are the “App Alert Tones.” Click on that option and you’ll get this screen:

App Alert Tones screen

We’re almost there (I know, right!?!). Now click on “Ring Alerts” and you’ll get this screen:

Here’s where you can adjust all of the alerts for your phone (through the Ring app) when someone pushes your doorbell. You can silence it. You can turn on or off the notifications. You can set it to Pop on screen. What we want is at the very bottom in the “Advanced” section. Click on that and you’ll get more options:

This is the same screen, just after I scrolled down to the see the advanced options.

Now, finally, we can change the sound. Click on “Sound” (it will indicate which sound you are currently using below “Sound”) and you’ll get this screen:

These are your options for doorbell notifications on your phone through the Ring App.

You can pick any of the sounds or music available there. If you want to set it to a song or something like that, you can put that into a folder on your phone called “Ringtones” and they will show up there.

Change Doorbell Sound on Amazone Echo Devices for Ring

In addition to changing the sound on your phone, you can also change the sound on your Amazon Echo if you have it connected to the Ring app. I’m not going to go through how to connect it to the Ring app as that is pretty straightforward (download the Ring skill for your Echo), but here is how to change your doorbell sound on your Amazon Echo.

First, open the Amazon Echo app:

Amazon Echo home screen.

In the bottom right, click on “Devices”:

A list of my devices.

I have a lot of devices set up with my Amazon Echo, so I actually have to scroll over to see all the devices (just swipe the list at the top to the left – Tinder style!) to see the option for All Devices:

I swiped left!

Click on “All Devices” and you’ll get a list of all the devices you have set up on Amazon’s Echo/Alexa app. You need to find the app that has a camera icon and is whatever you named your Ring video doorbell. Mine is called “Front Door”:

You’re looking for your Ring video doorbell in the list of devices.

Click on that and you’ll see the settings screen for your video doorbell:

The selected sound is under “Doorbell Sound.”

As you can see in the screenshot above, I had set up a “Howl” for Halloween. I want to switch it to something different. Click on the Doorbell Sound option and you’ll see a list of additional sounds:

The list of doorbell sounds.

The list includes seasonal options. I went with Xmas Elves. Select it and click back and that will be the new sound that is played through your Amazon Echo devices when the Ring video doorbell is pressed:

It worked!

Now, the next time I want to change this option, I won’t have to click on 50 different options in the various apps. Hooray for me (and you)!

Linux/Kubuntu – Disable Network Printer Auto Discovery

I don’t know when Kubuntu started automatically discovering printers on networks and then adding them to my list of printers, but it is a problematic feature in certain environments – like universities (where I work).

I set up my home printer on my laptop easy enough. But, whenever I open my laptop and connect to my work network, this feature searches for printers on the network and then adds them to my list of printers. I now have hundreds of printers that show up in my printers dialogue:

I didn’t manually add any of those printers. They were added automatically and are causing problems. First, it’s a pain in the ass to find the printer I want. Second, when I shutdown my computer, the OS has to run through all of those printers and make sure they are disconnected, which makes the OS hang for a couple of minutes every time I want to close down.

This is obviously a great idea in principle, but problematic in this environment.

So, how to turn this off. I found a solution. In a terminal, edit the following file:

sudo nano /etc/cups/cups-browsed.conf

In that file, you should just have to uncomment the following line (remove the hashtag ‘#’):

BrowseProtocols none

So, from this:

To this:

Afterward, try running the commands:

service cups-browsed restart
service cups restart

After making this change, my computer no longer automatically adds shared printers on my network. Hooray!

Unfortunately, making this edit did not remove all the shared printers it had already installed. I still had to remove them all manually, which was annoying. But at least they won’t be reinstalled automatically.

Linux – Failing to Read Encrypted DVDs

Thanks to the fine folks at VLC and Ubuntu, watching DVDs on Linux is generally pretty straightforward. Install “libdvd-pkg” and follow the prompts and you’re generally good to go. That works for me almost all of the time.

However, I recently tried to watch a DVD and had no luck. I would insert the DVD and then wait. With most other DVDs, after about 30 seconds, I’d get a prompt that my Kubuntu 19.04 system had read the DVD and I had several options to proceed (view the files, watch it in VLC, etc.). But with this one, my OS couldn’t even detect that there was a disk in the drive. I tried multiple approaches.

Here’s what K3b indicated:

When I tried loading it in VLC:

And from the command line in VLC:

The problem is not my drive. I regularly load disks in the drive and they work fine, including many disks that have CSS encryption that the libdvd-pkg addresses. But, try as I might, I could not get my computer to even recognize that there was a disk in the drive.

I have a blu-ray player connected to my home entertainment center. Worried that the disk may just be bad, I put it in the blu-ray player and it opened fine. That convinced me that the disk was using some form of encryption that is still not addressed in the libdvd-pkg. I did one final check. I inserted the disk into an old laptop I keep around that has Windows installed on it just to see if this really is a Windows vs. Linux thing. Sure enough, Windows immediately detected it and opened it right up.

After spending a good 5 hours or so trying to find a solution (including installing lots of packages and reading through dozens of threads in Linux forums), I didn’t find a solution. I wrote this post basically just to inform other Linux users that there are some DVDs out there that have encryption that prevent them from being opened in Linux. I’m running the latest version of Kubuntu as of this writing (19.04 beta) with all the suggested packages installed to examine a DVD. But, regardless of what I tried, my OS could not read this disk.

Update 2/3/2020:

A friendly reader (Fabian Echevarria) sent the following:

Last week I came across a DVD similar to that described in your article.

I was able to read the data, first using ddrescue to read to ISO. The resulting ISO also failed but now with a common Title 3 IFO error, which usually isn’t a deal breaker, but in this case continued to prevent play. So I read the raw directory (I normally use 7z/isoinfo) and then pulled each individual chapter from streams that seemed viable, in this case 59 through 72. Running md5sum on the resulting files determined that only two chapters differed between all those stream. A manual review of those two determined which were the correct chapters, which I combined into the final file. The combined file plays fine.

For more information on Fabian’s workarounds, see here.

Letter to the Editor: Government of the Corporations, by the Corporations, for the Corporations

While accepting “donations” for tens or hundreds of thousands of dollars from lobbyists, corporations, and special interests may technically be legal for Governor DeSantis, there is no question that is unethical. These are “bribes by another name.” And what they suggest is that our Governor is not the Governor of the people of Florida, but rather the Governor of Corporations based in Florida. Donald Trump has metaphorically set up a summer home in the “swamp” of corporate interests to personally enrich himself as President. Governor DeSantis is following his idol and has set up a metaphorical golf club in the swamp. Or, perhaps, by “draining the swamp” he really meant he was draining it right into his re-election coffers.

Letter to Stacy White, County Commissioner

Stacy White, Hillsborough County Commissioner, filed a lawsuit that is preventing Hillsborough County from improving its transportation infrastructure based on a ballot measure that was approved by 57% of voters in November 2018 (more about this here). Here’s the email I recently sent him:

Commissioner White,

I want to let you know that I’m really disappointed in you for filing the lawsuit that has held up improvements to our transportation system in Hillsborough County. I travel regularly for my work to conferences both in the US and around the world. In most major cities, when I arrive at the airport, I can easily hop on a train or subway and get to where I need to go.

I love Tampa’s airport. It’s my favorite airport perhaps in the world (though Zurich’s airport is pretty impressive as well). But I am appalled that we don’t have easy public transit from the airport to downtown Tampa. The transportation tax would have fixed that. And now, because of you, the entire process has been held up in stupid lawsuits.

I sit on the board of a professional organization that chooses where to hold conferences. One of the major factors in our decision-making process is public transit from the airport to the hotel. By not improving our transportation infrastructure, we are losing out on millions of dollars from such conferences every year. I hold you personally responsible for that.

I will be donating money to your opponent in the upcoming election to ensure you don’t have another term in office.

Sincerely,

Ryan Cragun
Hillsborough County Resident

(NOTE: You can submit your own letter here.)

Letter to the Editor: Ashley Moody’s mixed up priorities

I find it more than a little concerning that Florida Attorney General Ashley Moody wants to kill a petition drive to ban assault weapons (Ban Assault Weapons Now) but hasn’t said anything about the secretive effort to restrict constitutional amendments that is possibly being funded by utility companies in Florida. If Moody cared about Floridians and the law more than she did conservative politics and big donors, her actions would be exactly the opposite of what she is doing. She would be calling for the donors of the Keep Our Constitution Clean initiative to be revealed and would be encouraging reform to gun laws. Based on her actions, our chief law enforcement official in Florida is nothing more than a Republican team member defending corporate and Republican interests.

(NOTE: This is another letter I sent to the editor of the Tampa Bay Times that was not published.)