Restoring partial backups

Recently a client called quite desperate stating they had deleted some data and couldn’t find a way to get it back.
In our system we use this gem called paranoia for most of our models. This gem, overrides the default behaviour of destroy and instead of actually destroying the record, it sets a timestamp in the deleted_at attribute. It also adds a scope to the queries to exclude the deleted records by default. If you really want to delete the record, or search over deleted records, you can obviously do that as well.

Anyway, the problem would’ve been solved just by using this if we had the model in question configured to use paranoia. Unfortunately we didn’t as the initial requirements stated quite clearly that there was close to zero problems if some of the records were deleted. (*Lesson number one*: close to zero problem is not zero problem and space is not a current limitation, so when in doubt, save the data).

With the impossibility of using this gem’s skills, we had to turn to our backups. Our backups run on a daily basis during the night when the workload is lower. But when this problem was reported, much more data had been created which made it impossible to fully restore the backup and go on to our lives.

This way, we came up with a neat solution that got us back on track without much effort:
1. Download the most recent backup and load it into one of our machines.
We have a script that fetches a list of the existing backups and allows us to choose whether to create a new one at the execution time or use the last one generated. Since the data in production had been lost already we used the most recent backup before the report.
2. Create an auxiliary table that would include a subset of the data we wanted to restore and export it to a sql script.
This stackoverflow question turned out to be quite helpful for us to give us the idea of how to solve the problem.
3. Once the data was exported, we needed to edit the file to replace the auxiliary table name with the actual table name. Since the export does not write any drop statements but only inserts, it is safe to do so. So, we opened the file and replaced all the occurrences of the auxiliary table name by the actual table name. If the file is small enough you can easily open it in any text editor. If it turns out to be too big, try a scripting language to apply a sed in place.
4. Because our application is currently running on heroku machines, we still had to run this script in production to insert the missing data back. In order to do so, we used a simple bash command inspired from here
`cat file.sql | heroku pg:psql –app app_name`

cat outputs the contents of the file, and with | we redirect it to the command after it. In this case the command line interface with heroku.

This allowed us to insert the missing data successfully without messing with any other users data.

Emacs, next step

Once I’ve gotten familiar wiht the basic commands, my first problem was how ugly emacs looked out of the box. No one on earth can have any pleasure working on an ugly UI.
So I’ve started looking for how to change it.

This lead me to the initialization of configuring emacs defaults.


Vim or Emacs…. decisions decisions….

This will be a part of a series where I’ll be trying the devil’s question. Vim, or Emacs.
I do understand that for taking a real big advantage from any of both, you should be a long-term user, as it implies a lot of configuration and it takes a lot of personal taste.

Nevertheless, let the journey begin…

I’m currently using a Macbook Air with Yosemite in it.

Emacs… let’s do it.

Find and delete files in a tree

Recently I found that our current project had a bunch of temporary files that were not being cleaned up.
They are created by Emacs and not cleaned up, probably due to not closing the buffers properly.

As they were a bunch of them, I didn’t feel like doing one by one and they all matched a certain pattern:


So, to delete them, I’ve used find with args regex and delete.
Might become handy sometime in the future.

Bullet Gem

Recently in one of our projects, we decided to take a deeper look at N+1 queries, as well as general DB Operations performance. We’ve started by installing this awesome gem called bullet .
This gem, creates a log file for you, where it lists all the N+1 queries found, unnecessary eager loading queries and missing requires. This is quite useful as a starting point for improving your general web application performance.

We’ve successfully cleaned up most of the logged queries and we’ve moved on with our lives.
Because our application relies a lot on indexedDB, we’ve never experienced (at least not until today) a lot of hassle after making bullet part of our development environment.

Password Choosing

Recently came against the deal of choosing a secure password for some random access. As usual, I ran a bunch of generators looking for a decent idea as I usually don’t like the results of the generators. They generate too much rubish, impossible to memorize unless you use it frequently.

This time, though, I came accross this interesting xkcd strip that suggests a way to generate a strong password with little effort and easy to memorize.

xkcd password strip

Rails populate after migrate

Even though it is not a good policy to use the migrations to populate your database, sometimes, when there are very static things it is almost safe to assume that you can do it.
Nevertheless, if you are planning to run the migrations in batch (e.g develop, develop, develop, publish into production) you might run into problems that you haven’t in your dev environment.

The issue this time was regarding a migration that tried to populate a column previously created in another migration. It was failing when accessing it.

The issue became a bit weirder when we tried to print the attr_accessible of the Model itself, as well as the actual db columns and both outputs were showing that the column was there and supposedely accessible through the model.

Re-running the migration, after failing solved the issue which led me to think that the problem was somehow related with the environment loaded. I decided to throw in a debugger into the migration file and take a look to what was actually loaded. Happens that the migration, even though running successfully, it doesn’t force a reload of the changed Models. Therefore, our newly added column was not yet available.

Happily, there is a solution for this very specific problem:

This will force the reload of the model, making the newly column available on the next call. The use of this is in this scenario is also referenced in the documentation as the most common use for this method:

“The most common usage pattern for this method is probably in a migration, when
just after creating a table you want to populate it with some default values”

Please refer to:

See you

Multiple line wrapping CSS

Recently in our project we’ve had an issue that involved multiline text wrapping. It’s always been a bit shaddy to do it. Some people use character count and then chop the string, some others just rely on the height and so on.

The inexistence of a overall accepted answer worries me a little bit, given that it is not so uncommon.
I understand that, with the mobile invasion of the market, mobile first and changing the website to show contents according to which platform you are using makes total sense, and wrapping with ellipsis shouldn’t be the solution.

Hiding content from the users is never the solution, showing a bit and tell them “Ah! Ah! you can’t see more!!” is even worse! Nevertheless, in this specific scenario, wrapping with ellipsis is accepted provided that we can have multiple line.

Luckily for us, we are developing solely for google chrome, and we have webkit available, so here is the solution we’ve adopted.


IndexedDB does not create indexes on boolean

In my current job, we are using indexedDB to handle data offline and make it available on the browser even when there is no internet connection.

Recently, one of the tasks that we had made us think that would be useful to create an index with an id attribute and a boolean. The idea behind this was to filter the dataset returned by the indexedDB and avoid having to filter it.

Turns out, indexedDB does not create indexes over boolean values. If we reason it, booleans are actually a low cardinality field, therefore does not make much sense to create an index for a boolean column.
On the other side, using the boolean with something else, seems reasonable and useful for me.

We ended up by fetching all the results (filtered only by the attribute id) and then ignore the ones that have the boolean set to false. One other possible solution would be converting true/false to 1/0 but that doesn’t seem feasible in a real product situation.

In conclusion:

* IndexedDB does not create indexes for booleans columns
* If you try to create, it will appear on your list of indexes, but it won’t be filled at all
* If you try to access it, you’ll have an error thrown