Merge remote-tracking branch 'origin/staging' into staging

Conflicts:
	dashboard/config/videos.csv

Author: bcjordan

README.md

@@ -14,11 +14,11 @@ Many Windows developers have found that setting up an Ubuntu virtual machine is
 
 ## Install OS-specific prerequisites
 
-### OS X Mavericks
+### OS X Mavericks / Yosemite
 
-1. Install Homebrew: `ruby -e "$(curl -fsSL https://raw.github.com/Homebrew/homebrew/go/install)"`
-1. `brew install https://raw.github.com/quantiverge/homebrew-binary/pdftk/pdftk.rb enscript gs mysql imagemagick rbenv ruby-build`
-  1. If it complains about an old version of `X`, run `brew unlink X` and run `brew install X` again
+1. Install Homebrew: `ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"`
+1. `brew install https://raw.github.com/quantiverge/homebrew-binary/pdftk/pdftk.rb enscript gs mysql imagemagick rbenv ruby-build coreutils`
+  1. If it complains about an old version of `<package>`, run `brew unlink <package>` and run `brew install <package>` again
 1. Set up MySQL
   1. Have launchd start mysql at login: `ln -sfv /usr/local/opt/mysql/*.plist ~/Library/LaunchAgents`
   1. Start mysql now: `launchctl load ~/Library/LaunchAgents/homebrew.mxcl.mysql.plist`
@@ -33,27 +33,18 @@ Many Windows developers have found that setting up an Ubuntu virtual machine is
 
 1. `sudo aptitude update`
 1. `sudo aptitude upgrade`
-1. `sudo aptitude install -y git mysql-server mysql-client libmysqlclient-dev libxslt1-dev libssl-dev zlib1g-dev imagemagick libmagickcore-dev libmagickwand-dev nodejs openjdk-7-jre-headless libcairo2-dev libjpeg8-dev libpango1.0-dev libgif-dev curl`
+1. `sudo aptitude install -y git mysql-server mysql-client libmysqlclient-dev libxslt1-dev libssl-dev zlib1g-dev imagemagick libmagickcore-dev libmagickwand-dev nodejs npm openjdk-7-jre-headless libcairo2-dev libjpeg8-dev libpango1.0-dev libgif-dev curl pdftk`
   * **Hit enter and select default options for any configuration popups**
-1. `sudo aptitude install npm`
-1. `sudo ln -s /usr/bin/nodejs /usr/bin/node`
-1. `sudo npm update -g npm`
-1. `sudo npm install -g grunt-cli`
 
 ## Common setup
 
 1. `git clone https://github.com/code-dot-org/code-dot-org.git`
 1. `gem install bundler`
+1. `rbenv rehash` (if using rbenv)
 1. `cd code-dot-org/aws`
 1. `bundle install`
-1. `cd ../dashboard`
-1. `bundle install`
-1. `bundle exec rake db:create db:schema:load seed:all`
-1. `cd ../pegasus`
-1. `bundle install`
-1. `echo CREATE DATABASE pegasus_development | mysql -uroot`
-1. `rake db:migrate`
-1. `rake seed:migrate`
+1. `cd ..`
+1. `rake install`
 
 ## Organizational Structure
 
@@ -72,25 +63,47 @@ Our code is segmented into four parts:
 ## Running Dashboard
 
 1. `cd code-dot-org`
-2. `rake build:dashboard`
+2. `rake build:dashboard` (Generally, do this after each pull)
 3. `bin/dashboard-server`
 4. Visit [http://localhost.studio.code.org:3000/](http://localhost.studio.code.org:3000/)
 
 ## Running Pegasus
 
 1. `cd code-dot-org`
-2. `rake build:pegasus`
+2. `rake build:pegasus` (Generally, do this after each pull)
 3. `bin/pegasus-server`
 4. Visit [http://localhost.code.org:9393/](http://localhost.code.org:9393/)
 
 ## Building Blockly and Blockly-core (optional)
 
-The learn.code.org default dashboard install includes a static build of blockly, but if you want to make modifications to blockly or blockly-core:
+The learn.code.org default dashboard install includes a static build of blockly, but if you want to make modifications to blockly or blockly-core you'll want to enable building them in the build:
+
+### Enabling Blockly Builds
+
+You'll need to do this once:
+
+1. OS X:
+  1. Install the [Java 8 JDK](http://www.oracle.com/technetwork/java/javase/downloads/index.html)
+  1. Install [XQuartz](http://xquartz.macosforge.org/trac) (NOTE: This is required to build the Canvas dependency).
+1. `cd code-dot-org`
+1. Edit `locals.yml`
+  1. Add `build_blockly: true`
+  1. Add `build_blockly_core: true`
+  1. Add `use_my_blockly: true`
+1. `rake install`
+
+This configures your system to build blockly (and blockly-core) whenever you run `rake build` and to use the version of blockly that you build yourself.
+
+### Building Blockly and Blockly-Core
+
+1. `cd code-dot-org`
+1. `rake build`
+
+This will build everything you have set to build in `locals.yml`.
+
+You can use `rake build:blockly` and `rake build:blockly_core` to build a specific project.
 
-1. `cd code-dot-org/dashboard`
-1. `bundle exec rake 'blockly:dev[../blockly]'`
-  * This symlinks to dashboard reference the dev version of blockly
-1. Follow the blockly build instructions at `blockly/README` or blockly-core build instructions at `blockly-core/README`
+You can also set `build_dashboard: false` and/or `build_pegasus: false` in `locals.yml` if you don't need to build these frequently. They default to `true`.
 
 ## Contributing
 
@@ -146,7 +159,7 @@ Contributors should follow the GitHub [fork-and-pull model](https://help.github.
     - `git push origin branch_name`
 3. Go to the code-dot-org GitHub page
     - [https://github.com/code-dot-org/code-dot-org](https://github.com/code-dot-org/code-dot-org)
-4. For your submissinon to be reviewed
+4. For your submission to be reviewed
     - Click on the "Pull Request" link, look over and confirm your diff
     - Submit a pull request for your branch to be merged into staging
     - For bonus points, include screenshots in the description. Command + Ctrl + Shift + 4 in OS X lets you copy a screen selection to your clipboard, which GitHub will let you paste right into the description

Rakefile

@@ -1,4 +1,5 @@
 require_relative './deployment'
+require 'os'
 require 'cdo/hip_chat'
 require 'cdo/rake_utils'
 
@@ -178,6 +179,16 @@ namespace :install do
         blockly_build = CDO.use_my_blockly ? blockly_dir('build/package') : 'blockly-package'
         RakeUtils.ln_s blockly_build, dashboard_dir('public','blockly')
       end
+
+      if OS.linux?
+        RakeUtils.sudo_ln_s '/usr/bin/nodejs', '/usr/bin/node'
+        RakeUtils.sudo 'npm', 'update', '-g', 'npm'
+        RakeUtils.sudo 'npm', 'install', '-g', 'grunt-cli'
+      elsif OS.mac?
+        RakeUtils.system 'brew install node'
+        RakeUtils.system 'npm', 'update', '-g', 'npm'
+        RakeUtils.system 'npm', 'install', '-g', 'grunt-cli'
+      end
     end
   end
 

bin/activity-monitor

@@ -1,10 +1,12 @@
 #!/usr/bin/env ruby
-require 'csv'
-require_relative '../deployment'
+# including dashboard environment doesn't work if not in the dashboard dir
+Dir.chdir(File.expand_path('../../dashboard', __FILE__))
+require_relative '../dashboard/config/environment'
 require 'cdo/only_one'
 require 'cdo/hip_chat'
 require 'cdo/user_agent_parser'
 
+
 FiveMinutes = 5 * 60 * 1000
 OneHour = 60 * 60 * 1000
 OneDay = 24 * OneHour
@@ -37,7 +39,8 @@ HardLevelAttemptsPerHour = {
   1108 => 10,
   1110 => 10,
   1430 => 10,
-  1434 => 10
+  1434 => 10,
+  1441 => 10
 }
 
 MinimumAttemptsPerHour = 5
@@ -141,6 +144,26 @@ class Monitor
 
 end
 
+BrowserToBrowserStack = {
+  'IE 11.0' => {os: 'Windows', os_version: '8.1', browser: 'IE', browser_version: '11.0'},
+  'IE 10.0' => {os: 'Windows', os_version: '7', browser: 'IE', browser_version: '10.0'},
+  'IE 9.0' => {os: 'Windows', os_version: '7', browser: 'IE', browser_version: '9.0'},
+  'Chrome' => {os: 'Windows', os_version: '8.1', browser: 'Chrome', browser_version: '39.0'},
+  'Firefox' => {os: 'Windows', os_version: '8.1', browser: 'Firefox', browser_version: '33.0'},
+  'Safari' => {os: 'OS X', os_version: 'Yosemite', browser: 'Safari', browser_version: '8.0'},
+  'Android' => {os: 'android', os_version: '4.4', device: 'Samsung Galaxy S5'},
+  'Mobile Safari' => {os: 'ios', os_version: '8.0', device: 'iPad Air'},
+  'Opera Mini' => {os: 'opera', device: 'Samsung Galaxy Tab'}
+}
+
+def browserstack_url(browser, destination_url)
+  params = BrowserToBrowserStack[browser]
+  return nil unless params
+
+  # eg: http://www.browserstack.com/start#os=Windows&os_version=7&browser=IE&browser_version=11.0&zoom_to_fit=true&full_screen=true&autofit=true&url=studio.code.org&resolution=2048x1536&speed=1
+  "http://www.browserstack.com/start#" + URI.encode_www_form(params.merge(resolution: '2048x1536', url: destination_url))
+end
+
 def main()
   now = DateTime.now.new_offset(0)
 
@@ -164,12 +187,18 @@ def main()
       rate = starts == 0 ? 1.0 : finishes.to_f / starts.to_f
 
       if status == :fail
-        log_string =
-          "<b>#{i[:browser]}</b> hasn't passed <b><a href=\"http://learn.code.org/levels/#{i[:level]}\">level #{i[:level]}</a></b> despite <b>#{i[:starts]}</b> attempts in the last #{hours} hour(s)."
+        level_url = "http://learn.code.org/levels/#{i[:level]}"
+
+        log_string = %Q[<b>#{i[:browser]}</b> hasn't passed <b><a href="#{level_url}">level #{i[:level]}</a></b> despite <b>#{i[:starts]}</b> attempts in the last #{hours} hour(s)]
+        sl = Level.find(i[:level]).script_levels.first
+        if sl
+          level_url = "http://studio.code.org/s/#{sl.script.id}/level/#{sl.id}"
+        end
+        log_string += %Q[ <a href="#{browserstack_url(i[:browser], level_url) || level_url}">&#x2708; Repro It</a>]
         HipChat.developers log_string, color:'red', notify:1
         HipChat.log log_string, color:'red'
       end
-    end
+  end
 end
 
 main if only_one_running?(__FILE__)

bin/force-gsheet

@@ -1,5 +1,5 @@
 #!/usr/bin/env ruby
-require '../deployment'
+require File.expand_path('../../deployment', __FILE__)
 require 'cdo/db'
 
 def main()

bin/upload-logs-to-s3

@@ -3,7 +3,7 @@ require_relative '../deployment'
 
 def upload_logs(application)
   raise 'Unknown application.' unless ['dashboard', 'pegasus'].include? application
-  system "aws s3 sync #{deploy_dir(application, 'log')} s3://cdo-logs/hosts/#{CDO.name}/#{application} --exclude .gitignore"
+  system "aws s3 sync #{deploy_dir(application, 'log')} s3://cdo-logs/hosts/#{CDO.name}/#{application} --exclude .gitignore 2>&1 > #{deploy_dir(application, 'log', 'uploads-logs-to-s3.log')}"
 end
 
 def main