All articles

  1. Om, Clojurescript, and Testing

    2015-07-16

    This past week I started learning React, Om, and Clojurescript all at once. When beginning to use cemerick's .clojurescript.test, I kept running into this error:

    1
    2
    3
    4
    5
    6
    7
    Error: cemerick is undefined
    
    ERROR: cemerick.cljs.test was not required.
    
    You can resolve this issue by ensuring [cemerick.cljs.test] appears
    in the :require clause of your test suite namespaces.
    Also make sure that your build has actually included any test files.
    

    But I clearly had included it in my test! I googled and grumbled, but could not figure out what was wrong. Finally I discovered that slimerjs has the -jsconsole flag, which, as the docs say, will

    1
    Open a window to view all javascript errors during the execution
    

    Great, using that I finally found the actual problem:

    1
    2
    3
    4
    Script Error: Error: Assert failed: No target specified to om.core/root
    (not (nil? target))
           Stack:
             -> file:///tmp/runner6386761518784950059.js.html: 55456
    

    This makes much more sense. The issue is that my core.cljs namespace was running om/root when the page loads. The code looked like:

    1
    2
    3
    (om/root main-view
             app-state
             {:target (. js/document (getElementById "app"))}))
    

    But since the tests are not loading the index.html page (as they shouldn't), there is no element with ID app. Ultimately the problem is with running code at the namespace level. What would be preferred would be if there were some way to specify a main function to initialize the app. This would be run for the actual application, but not the tests.

    First take at a Solution

    It took awhile of searching, but I finally found some inspiration from this project and specifically this line of code:

    1
    <script type="text/javascript">goog.require("react_tutorial_om.app");</script>
    

    I realized I could just wrap my om/root call in a main function and then call this from the index.html page. Here is what the code in core.cljs looks like now:

    1
    2
    3
    4
    (defn app []
      (om/root main-view
               app-state
               {:target (. js/document (getElementById "app"))}))
    

    and the corresponding code in index.html:

    1
    2
    3
    4
    <script type="text/javascript">
    goog.require("my.namespace.core");
    my.namespace.core.app();
    </script>
    

    Now running the tests no longer had any problem. However, I realized that lein figwheel was not reloading the page properly when I made changes to the code. This is because the javascript would be reloaded, which previously was running om/root every time. To solve this I added to the on-js-reload function so that the app was reinitialized:

    1
    2
    (defn on-js-reload []
      (app))
    

    Improving the Solution

    As I continued learning about Om, I came across the om-cookbook repository. The following is based on the structure of the project in the recipes/routing-with-secretary directory (and possibly others in the repo).

    Let's assume your project currently has this directory structure:

    resources/...
    src/my/namespace/core.cljs
    project.clj
    

    We are going to add a directory called env which will house code that is specific to different environments, namely development vs. production. Create directories such that your project now looks like this:

    env/dev/src/my/namespace/dev.cljs
       /prod/src/my/namespace/prod.cljs
    resources/...
    src/my/namespace/core.cljs
    project.clj
    

    You can see that in env/dev and env/prod we mimick the src directory. Within dev.cljs we will add code that is only to be run when developing. Here is what that namespace will basically look like for the dev environment:

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    (ns my.namespace.dev
      (:require [my.namespace.core :as core]
                [figwheel.client :as figwheel :include-macros true]))
    
    (enable-console-print!)
    
    (defn on-js-reload []
      (core/app))
    
    (core/app)
    

    For production, this can be much simpler:

    1
    2
    3
    4
    (ns my.namespace.prod
      (:require [my.namespace.core :as core]))
    
    (core/app)
    

    Now all we need to do is modify project.clj to use these environments. This is accomplished using different build configurations. Here is a sample of how that would look:

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    :cljsbuild {:builds [{:id "dev"
                          :source-paths ["src" "env/dev/src"]
                          ; blah blah blah
                          }
                         {:id "prod"
                          :source-paths ["src" "env/prod/src"]
                          ; blah blah blah
                          }
                         {:id "test"
                          :source-paths ["src" "test"]
                          ; blah blah blah
                          }]}
    

    Finally, make sure to remove the code that was added to index.html in our first take at a solution.

    And there you have it. The dev environment will end up compiling dev.cljs, and since this namespace includes a call to core/app at the namespace-level, it will run when the javascript is loaded. We do not include the file for the test build, meaning the tests do not try to run om/root.

    Alternative Approach

    An entirely different approach to all of this (and perhaps a lot simpler) would be to simply check if your target element exists. Your code could then look like

    1
    2
    3
    4
    (if-let [target (. js/document (getElementById "app"))]
      (om/root main-view
               app-state
               {:target target}))
    

    You might prefer this approach. The reason I tend to not like this is that you still have functionality that executes when you require the namespace. It also introduces a silent failure. If you change the main element to have a different id, your app would just show up blank with no errors printed.

    I was pretty surprised to not be able to find anything about this. Maybe I'm missing something obvious. I am new to Clojurescript and Om, so it could just be a newbie mistake. If so let me know!

  2. Organizing photos with bash

    2015-01-02

    I've been trying to organize my photos recently. I hadn't kept up with them and I recently discovered I have a pile of photos dumped into a single directory. I like to separate photos by the day they were taken so I end up with directories like:

    2015-01-01_new-years-day
    2014-12-31_new-years-eve
    2014-12-25_christmas
    ...
    

    Wanting to organize these as painlessly as I could think, I sought to run a few bash commands to split my pile of photos into directories like these. The first issue I ran into was the file formats - these were raw photos. I had found a nice tool called exif (sudo apt-get install exif) which I could use to extract the date the pictures were taken, but it only works on JPG images.

    So step one was to convert the files. The ImageMagick tool (sudo apt-get install imagemagick) has a command called convert that will do just that. To convert them all without risking overwriting anything, a simple for loop works nicely:

    for f in *.NEF; do
        convert $f $f.jpg
    done
    

    This will convert files like DSC_0001.NEF to DSC_0001.NEF.jpg. Running this command was taking awhile and I wanted to see how much work was left to do. watch is the perfect tool for the job. It lets you repeatedly run a command and see its output. So all I needed to do was write a command that would list all of the .NEF files that did not have corresponding .NEF.jpg files. To do that, I used:

    for f in *.NEF; do
        ls $f.jpg &>/dev/null || echo $f
    done
    

    I ignored all output of ls (&>/dev/null) because the return code would be enough to determine whether the file existed. More importantly, I didn't want to have to parse through whatever output the command threw at me. The || echo $f will only be executed if the ls failed, indicating the file doesn't exist.

    I had issues running this directly with watch, but I didn't want to spend time debugging. Instead I saved the command to a file and ran:

    chmod +x script.sh
    watch ./script.sh
    

    Every 2 seconds the screen updated letting me know how many files were left to convert.

    While the conversion was running, I prepared to get exif data out of the jpg files. I mentioned above that exif will do just that. Here's what a sample run looked like:

      $ exif DSC_0093.JPG 
    EXIF tags in 'DSC_0093.JPG' ('Motorola' byte order):
    --------------------+----------------------------------------------------------
    Tag                 |Value
    --------------------+----------------------------------------------------------
    Manufacturer        |
    Model               |
    X-Resolution        |500.0000000
    Y-Resolution        |500.0000000
    Resolution Unit     |Inch
    Software            |Picasa 3.0
    Date and Time       |2007:04:09 16:15:47
    YCbCr Positioning   |Centered
    Padding             |2060 bytes undefined data
    Compression         |JPEG compression
    X-Resolution        |72
    Y-Resolution        |72
    Resolution Unit     |Inch
    Exif Version        |Exif Version 2.2
    Date and Time (Origi|2007:12:25 11:07:52
    Date and Time (Digit|2007:12:25 11:07:52
    Components Configura|Y Cb Cr -
    User Comment        |
    Sub-second Time (Ori|00
    Sub-second Time (Dig|00
    FlashPixVersion     |FlashPix Version 1.0
    Color Space         |Internal error (unknown value 65535)
    Pixel X Dimension   |3008
    Pixel Y Dimension   |2000
    Custom Rendered     |Custom process
    Image Unique ID     |517d7ada5c38d78cd60a3032343e64f1
    Padding             |2060 bytes undefined data
    Interoperability Ver|0100
    --------------------+----------------------------------------------------------
    EXIF data contains a thumbnail (5597 bytes).
    

    (This was a really old photo - most of them were much more recent.)

    I noticed there was the Date and Time tag in there. And in reading the man page for exif, saw that I could ask for values of specific tags using the --tag argument. Here is what happened for the same file above:

      $ exif --tag 'Date and Time' DSC_0093.JPG 
    EXIF entry 'Date and Time' (0x132, 'DateTime') exists in IFD '0':
    Tag: 0x132 ('DateTime')
      Format: 2 ('ASCII')
      Components: 20
      Size: 20
      Value: 2007:04:09 16:15:47
    

    We can work with that! A few grep and awk commands later and I had:

      $ exif --tag 'Date and Time' DSC_0093.JPG \
        | grep Value \
        | awk '{print $2}' \
        | awk -F':' '{print $1 "-" $2 "-" $3}'
    2007-04-09
    

    To break that down: grep Value is pretty obvious. It will leave only the line containing Value so now our result will look like

      Value: 2007:04:09 16:15:47
    

    Then this gets piped to awk '{print $2}'. This is a very common type of awk command to run. Since awk splits on whitespace by default, this will get us the second column, which in this case contains the text

    2007:04:09
    

    I didn't care about the time of the photo since I'm planning to lump all of the photos from the same day into a given directory. If I had wanted the date and time, I could have used awk '{print $2 " " $3}'.

    Now that I have the date, I want it in my preferred format of yyyy-mm-dd so I once again turned to awk. This time I used -F':' to specify that I wanted to split on : instead of whitespace. Then I rest of the command glues the date, month, and year parts together with - in between.

    So now I have to use all of this to make the directories and dump files into them. Before actually creating any directories or moving any files, however, I wanted to sanity check everything. I wrote a loop that spit out each of the *NEF.jpg files and the date I calculated for them:

    for f in *NEF.jpg; do
        echo -n "$f "
        dt=$(exif --tag 'Date and Time' $f \
                | grep Value \
                | awk '{print $2}' \
                | awk -F':' '{print $1 "-" $2 "-" $3}')
        echo $dt
    done
    

    The -n flag to echo tells it to not print a newline. I then saved the result of the command to calculate the picture's date to a variable called dt so that I could reuse it in the code to follow. Here's a snippet of the results:

    DSC_1455.NEF.jpg    2014-07-28
    DSC_1456.NEF.jpg    2014-07-28
    DSC_1457.NEF.jpg    2014-07-28
    DSC_1458.NEF.jpg    2014-07-28
    DSC_1460.NEF.jpg    2014-07-28
    DSC_1461.NEF.jpg    2014-07-28
    DSC_1463.NEF.jpg    2014-07-28
    DSC_1506.NEF.jpg    2014-09-03
    DSC_1507.NEF.jpg    2014-09-03
    DSC_1512.NEF.jpg    2014-09-03
    

    After sanity checking the results, I felt confident enough to go forward. Here is the final product:

    for f in *NEF.jpg; do
        dt=$(exif --tag 'Date and Time' $f \
                | grep Value \
                | awk '{print $2}' \
                | awk -F':' '{print $1 "-" $2 "-" $3}')
    
        mkdir $dt &>/dev/null
        mv ${f%.jpg} $dt/
    done
    

    The most interesting part of this code is the ${f%.jpg}. This chops .jpg off of the end of the string. To fully grasp what's going on, check out this article, specifically the section titled Substring Removal.

    And that was it! Was a lot of faster than doing it by hand. Maybe there are tools out there that could have done this for me, but I found this more fun. You can really do some neat stuff with everyday command-line tools. I highly recommend using them for tasks like this to become comfortable writing bash.

  3. Manipulating Vim Registers

    2014-11-02

    Recently I came across PHP code that looked similar to this:

    1
    2
    3
    4
    5
    6
    7
    const YEAR = 1;
    const QUARTER = 2;
    const MONTH = 3;
    const WEEK = 4;
    const DAY = 5;
    const HOUR = 6;
    const SECOND = 7;
    

    I wanted to write a switch statement which handled each of those constants. This is what each case statement would look like:

    1
    2
    3
    4
    5
    6
    7
    case self::YEAR:
    case self::QUARTER:
    case self::MONTH:
    case self::WEEK:
    case self::DAY:
    case self::HOUR:
    case self::SECOND:
    

    One way to achieve this would be to yank the whole text, paste it where the switch will live, and with the help of a macro, change each line into a case statement. This article is a great reference for advanced macro techniques, any of which would be useful if we took this approach. However, it's possible to be more succinct by building up the case statements incrementally with the help of registers (:help registers). This post will show a few ways to do that so that you can learn how to master registers. Here are some examples to start us off:

    Command Explanation
    :let @a = "" Clear out register "a"
    :let @a .= "I am register a!" Append the string "I am register a!" to register "a"
    :let @c = "b is: " . @b Set register "c" to be the string "b is: " concatenated with register "b"
    :reg c Check what's in register "c"
    "dyy Replace register "d" with the current line
    "Dyy If you'd rather not overwrite register "d", but instead append to it

    For more details on the :let @ command, check out :help :let-@.

    Knowing just the above will get us far. Let's see how we can apply them toward making our case statements. Our first exercise is pretty simple. We will get the contents of register a to be:

    1
    2
    3
    4
    5
    6
    7
    YEAR
    QUARTER
    MONTH
    WEEK
    DAY
    HOUR
    SECOND
    

    (note that there will actually be an empty first line)

    Start by moving your cursor to the word "YEAR" and initializing register a to a newline:

    1
    2
    /YEAR<CR>
    :let @a = "\n"<CR>
    

    Doing so will make register a linewise (:help linewise). I explain below why this happens and what effect is has, but for now, suffice it to say that when you yank more text into a, Vim will keep a newline at the end of the register. Go ahead and yank the word "YEAR" onto the end of register a to see it happen:

    1
    "Aye
    

    As we saw above, "A tells vim that the next thing we yank should be appended to register a and ye means to yank until the end of the word. Examine register a with :reg a to see it is set to^JYEAR^J (^J stands for the newline character). Move your cursor down so it is on QUARTER and again use "Aye to append onto a. a is now ^JYEAR^JQUARTER^J. Do this for every line (or even better, record a macro to automate it) and register a will have the text we were going for.

    That was a fun exercise, but to finish writing each case statement, we would need to edit each line in register a, which we wanted to avoid in the first place. We can do better. As before, move your cursor to YEAR, but this time clear out register a entirely:

    1
    2
    /YEAR<CR>
    :let @a = ""<CR>
    

    We will use another register to help us. Yank YEAR into register b and use it to append to register a:

    1
    2
    "bye
    :let @a .= "case self::" . @b . ":\n"<CR>
    

    This concatenates the text case self:: with the contents of register b and the string :\n. Register a is not linewise this time, so we have to append the newline ourselves. a will now be case self::YEAR:^J. If we repeat the above for each line (once again - use a macro!), register a will be:

    1
    2
    3
    4
    5
    6
    7
    case self::YEAR:
    case self::QUARTER:
    case self::MONTH:
    case self::WEEK:
    case self::DAY:
    case self::HOUR:
    case self::SECOND:
    

    This is perfect. We eliminated the need to do any work after we pasted the results. Instead, we just incrementally built the desired text within register a.

    We can make one further improvement. Using register b was an interesting way of showing how you can use different registers and concatenate them together to make the finished product. It's unnecessary, however, since we can use <C-R><C-W> (:help <c_CTRL-R_CTRL-W>) to insert the word under the cursor each time we append to a. To demonstrate this, again move to the beginning of YEAR and clear register a. This time, skip yanking to register b and do:

    1
    :let @a .= "case self::<C-R><C-W>:\n"<CR>
    

    After doing this for each line, register a will be the same as before.

    Linewise vs. characterwise registers

    As I mentioned above, Vim will make a register linewise when the expression you assign using :let @... ends in a newline. Vim will keep a newline at the end of the register whenever you append to it by yanking, deleting, etc., but not when you directly modify the register using :let @.... Even if you already have something in the register and append using :let @a .= ..., the register will change to linewise.

    If you want to get around this, you have a few options. The first uses yanking to get a newline at the end of your register. In insert mode, type <C-V><C-M>. This inserts a newline character which you can yank into register a with "ayl. Register a will still be characterwise, but a will have a trailing newline character.

    A second option, which is perhaps a bit hackier, is to not append just a newline to your register, but a newline plus some other character. For example:

    1
    :let @a = "\n|"<CR>
    

    Here the register ends with the pipe character. You could use a space or tab or any non-newline character. In our PHP example from the previous section, this technique might have been useful if we wanted each line to begin with a tab character, for example.

  4. Automatically Focus on Amazon's Search Bar

    2013-03-07

    Whenever I visit Amazon's homepage, I do so hoping to search for things. I'm not one to browse around and I'm sure many others are the same. However, I get annoyed each time this happens because the search bar does not have the focus when you visit the homepage. After coming across a comment on Hacker News describing the same problem, I realized I could fix this with a pretty simple Chrome extension. Check it out here and read on for a short description of how I put it together.

    About one or two years back, I was interested in making extensions for Chrome, but never went very far with it. However, the brief foray meant that yesterday, I knew the basics I would need to make the extension. The most important part of this is the manifest, which describes your extension, lists the permissions it needs, files it contains, and so forth.

    The manifest for this app was really simple:

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    {
        "manifest_version": 2,
        "name": "Amazon Search AutoFocus",
        "description": "Automatically focus on search bar when visiting Amazon",
        "version": "1",
        "permissions": [
            "http://*.amazon.com/",
            "https://*.amazon.com/"
        ],
        "content_scripts": [
            {
                "matches": ["http://*.amazon.com/"],
                "js": ["script.js"]
            }
        ]
    }
    

    Most of this is pretty obvious. The documentation on the manifest_version indicates that version 1 is deprecate. The version indicated lets Google manage the extension among users. This number will be used to determine if a user is on an older version of the extension and it needs to be updated.

    Permissions are bit complicated, as they can include whether the extension can operate on tabs or windows, have access to sites, etc. For this really simple app, I only need to indicate that I want access to the amazon sites.

    Finally, the hardest part of it all was figuring out how to get some javascript to run when I visited Amazon's homepage. Using some Google-foo, I discovered that Content Scripts are the way to do this. As the documentation says, content scripts run in the context of web pages. This means the javascript file specified above (script.js) will be run when Amazon loads. This is exactly what I wanted!

    So finally I just needed to write the javascript. It turned out to be dead simple. Here's the only line needed:

    document.getElementById("twotabsearchtextbox").focus();
    

    I briefly examined Amazon's search box using the Chrome developer tools and I saw that the ID of the search box was twotabsearchtextbox. So I used some native javascript functions to set the focus and it was done!

    Hopefully this helps if you want to write your own Chrome extension! If I make any changes or improvements to this, the updated code will be on Bitbucket for your viewing pleasure.

Page 1 / 1