Cacomania: tag

Cacomania

Kill all long running MongoDB queries at once

Guido Krömer - 1093 days ago - Tags: ,

If of some very running queries killing your MongoDB server you should try to execute the following piece of code in the shell. It kills all long running queries at once. You can define the maximum execution time by using the first parameter, which is set to 120 seconds by default.

(function (sec) {db.currentOp()['inprog'].forEach(function (query) { 
    if (query.op !== 'query') { return; } 
    if (query.secs_running < sec) { return; }  

    print(['Killing query:', query.opid, 
           'which was running:', query.secs_running, 'sec.'].join(' '));
    db.killOp(query.opid);
})})(120 /*The maximum execution time!*/);

This is the output generated by my little emergency helper:

Killing query: 598841625  which was running: 1892 sec.
Killing query: 598818776 which was running: 1931 sec.
Killing query: 598771338 which was running: 2048 sec.
Killing query: 577527534 which was running: 73569 sec.
Killing query: 577589300 which was running: 73404 sec.
Killing query: 577540188 which was running: 73535 sec.
Killing query: 598254342 which was running: 3114 sec.
Killing query: 597396058 which was running: 4811 sec.
Killing query: 597609629 which was running: 4382 sec.
Killing query: 597403975 which was running: 4801 sec.
Killing query: 597423364 which was running: 4766 sec.
Killing query: 597389768 which was running: 4842 sec.

I hope my little posting helped you, please let me know if you liked or disliked it.

Comment

MongoDB with PHP "… BadValue $in needs an array"

Guido Krömer - 1152 days ago - Tags: ,

Recently I had a problem when using MongoDB with PHP using the $in operator which ends up with the following error: "Can't canonicalize query: BadValue $in needs an array".

I did a simple query and used the $in operator thousand times before, but what went wrong? Let's take a look at the simplified code below, an indexed array of tags is given to filter for blog posts in a collection, but before passing the tags to the query I removed invalid tags from the array using unset().

foreach ($tags as $key => $tag) {
    if (!valid($tag)) {
        unset($tags[$key]);
    }
}

$posts = $blogPostCollection->find(['Tag' => ['$in' => $tags]]);

Using unset was the problem, but why? When iterating over an array after unsetting a value everything seems to be right:

php > $ids = ['a', 'b', 'c', 'd'];
php > unset($ids[1]);
php > foreach ($ids as $id) { echo "$id "; }
a c d 

The problem is that unset() produces holes in an indexed array, c has still the index two… :

php > $ids = ['a', 'b', 'c', 'd'];
php > var_export($ids);
array (
  0 => 'a',
  1 => 'b',
  2 => 'c',
  3 => 'd',
)
php > unset($ids[1]);
php > var_export($ids);
array (
  0 => 'a',
  2 => 'c',
  3 => 'd',
)

One solution is using array_splice() instead of unset() which prevents holes in an index array:

php > $ids = ['a', 'b', 'c', 'd'];
php > array_splice($ids, 1, 1);
php > var_export($ids);
array (
  0 => 'a',
  1 => 'c',
  2 => 'd',
)

Or you could use array_values() on the array after an unset operation which creates an new array containing all values from the origin array.

php >$ids = ['a', 'b', 'c', 'd'];
php > unset($ids[1]);
php > var_export(array_values($ids));
array (
  0 => 'a',
  1 => 'c',
  2 => 'd',
)

I hope this little post helped you with this or a similar problem, feel free to leave a comment.

Comment

MongoDB: manipulating values if a "update" is not enough

Guido Krömer - 1806 days ago - Tags: , ,

Sometimes the MongoDB update function does not fit the requirements. Since JavaScript can be executed directly in the database (if it's not disabled) it can be used for more complex updates, for example. This happens without moving the data to the application and back to the database.

This small example does the same as: myColl.update({myValue: 1}, {$inc: {myValue: 1}})

db.myCollection.find({myValue: 1}).forEach(function(doc){ 
    var newValue = doc.myValue + 1;
    db.site.update({_id: doc._id}, {$set: {myValue: newValue}})  
})

This is a concrete example where tags have been saved as a comma separated string. Creating an index, for fast searching in the tags, on this field would not make sense. Converting this string field into an array of tags should solve the problem. This piece of code does this job for each site which has been tagged, for speeding up only the tags field and MongoDB's _id field gets queried from those documents.

db.sites.find({tags: { $exists: true }}, {tags: 1}).forEach(function(doc){ 
   db.site.update({_id: doc._id}, {$set: {tags : doc.tags.split(',')}})  
})

Comment