Showing posts with label recaptcha. Show all posts
Showing posts with label recaptcha. Show all posts

Monday, July 11, 2011

Google recaptcha with CSP, hosting javascript locally

I display google's recaptcha from google's recaptcha javascript hosted on my own server after a little manipulation (since I have CSP turned on, I have to bypass the in-body script and setInterval). But I still need to ping google every time for a new challenge.

1. So I saved my domain's equivalent of the RecaptchaState javascript medai/js/google/recState.js.

2. Refresh this file in views.py's by calling this function:
def recaptchaRefresh():
    # get the Recaptcha state.
    url = "https://www.google.com/recaptcha/api/challenge?k=%s" % settings.RECAPTCHA_PUBLIC_KEY
    resock = urllib.urlopen(url)
    data = resock.read()
    resock.close()

    # extract the recaptcha state part of the string
    docloc = data.find("document.write")

    recaptchaState = data[:docloc]

    f = open('media/js/google/recState.js', 'r+')
    f.write(recaptchaState)
    f.close()

3. Copy paste recaptcha.js and make the following changes for CSP compatibility:

3.1 CSP blocks setIntervals that takes string parameters, so change it into a function:
//Recaptcha.timer_id = setInterval("Recaptcha.reload('t');", (a.timeout - 300) * 1E3) 
// -->
Recaptcha.timer_id = setInterval( function() {Recaptcha.reload('t'); }, (a.timeout - 300) * 1E3)

3.2 CSP blocks in-body javascript, host it externally
//} else document.write('<div id="recaptcha_widget_div" style="display:none"></div>'), document.write('<script>Recaptcha.widget = Recaptcha.$("recaptcha_widget_div"); Recaptcha.challenge_callback();<\/script>');
// -->
} else document.write('<div id="recaptcha_widget_div" style="display:none"></div>'), document.write('<script src="http://haoqili.scripts.mit.edu/js/test3.js"><\/script>');

where http://haoqili.scripts.mit.edu/js/test3.js has "Recaptcha.widget = Recaptcha.$("recaptcha_widget_div"); Recaptcha.challenge_callback();"

4. and in your template.html include the javascript from step 2 and 3.

5. Change your settings.py's CSP policies to have "http[s]://www.google.com" allowed in many places. See example

Recaptcha Hunt

The hunt to figure out how to do recaptcha with CSP is on its 4th day. A few things learned:

--> don't assume anything. I could have figured out the CSP_REPORT_ONLY if I actually tried amo's settings word by word.
--> copy paste the entire chunk

--> read and try to make sence of error messages. think about them.

Sunday, July 10, 2011

How ReCaptcha should not work with in-body script

Big Question: Why doesn't my ReCaptchaField show up while AMO's ReCaptcha show up if our ReCaptchaField stuff matches exactly, including the parts that displays the in-body javascript? Issue on github

Skip to solution

How an In-Body Javascript is ultimately introduced in Google Recaptcha:

1. Your Django Form has ReCaptchaField. e.g. File: apps/users/forms.py
import captcha.fields

class UserRegisterForm(happyforms.ModelForm, PasswordMixin):
    passwords ...  
               
    recaptcha = captcha.fields.ReCaptchaField()

    ... irrelevent stuff ...

2. captcha.fields.ReCaptchaField() in zamboni/vendors (not shown on zamboni github), but it's on Mozilla's django-recaptcha
from django.conf import settings
from django import forms
from django.utils.encoding import smart_unicode
from django.utils.translation import ugettext_lazy as _

from recaptcha.client import captcha

from captcha.widgets import ReCaptcha

class ReCaptchaField(forms.CharField):

    default_error_messages = {
        'captcha_invalid': _(u'Invalid captcha')
    }

    def __init__(self, *args, **kwargs):
        self.widget = ReCaptcha
        self.required = True
        super(ReCaptchaField, self).__init__(*args, **kwargs)

    def clean(self, values):
        super(ReCaptchaField, self).clean(values[1])
        recaptcha_challenge_value = smart_unicode(values[0])
        recaptcha_response_value = smart_unicode(values[1])
        check_captcha = captcha.submit(recaptcha_challenge_value,
            recaptcha_response_value, settings.RECAPTCHA_PRIVATE_KEY, {})
        if not check_captcha.is_valid:
            raise forms.util.ValidationError(
                    self.error_messages['captcha_invalid'])
        return values[0]

3. from captcha.widgets import ReCaptcha
from django import forms
from django.utils.safestring import mark_safe
from django.conf import settings
from recaptcha.client import captcha

class ReCaptcha(forms.widgets.Widget):
    recaptcha_challenge_name = 'recaptcha_challenge_field'
    recaptcha_response_name = 'recaptcha_response_field'

    def render(self, name, value, attrs=None):
        use_ssl = False
        if 'RECAPTCHA_USE_SSL' in settings.__members__:
            use_ssl = settings.RECAPTCHA_USE_SSL
        return mark_safe(u'%s' %
                         captcha.displayhtml(settings.RECAPTCHA_PUBLIC_KEY,
                                             use_ssl=use_ssl))
     ...

4. from recaptcha.client (which is from Python's recaptcha client) import captcha
API_SSL_SERVER="https://api-secure.recaptcha.net"
API_SERVER="http://api.recaptcha.net"

def displayhtml (public_key,
                 use_ssl = False,
                 error = None):
    """Gets the HTML to display for reCAPTCHA

    public_key -- The public api key
    use_ssl -- Should the request be sent over ssl?
    error -- An error message to display (from RecaptchaResponse.error_code)"""

    error_param = ''
    if error:
        error_param = '&error=%s' % error

    if use_ssl:
        server = API_SSL_SERVER
    else:
        server = API_SERVER

    return """<script type="text/javascript" src="%(ApiServer)s/challenge?k=%(PublicKey)s%(ErrorParam)s"></script> # this src contains in-body script!!

<noscript>
  <iframe src="%(ApiServer)s/noscript?k=%(PublicKey)s%(ErrorParam)s" height="300" width="500" frameborder="0"></iframe><br />
  <textarea name="recaptcha_challenge_field" rows="3" cols="40"></textarea>
  <input type='hidden' name='recaptcha_response_field' value='manual_challenge' />
</noscript>
""" % { 
        'ApiServer' : server,
        'PublicKey' : public_key,
        'ErrorParam' : error_param,
        }   

5. The src directs to (key varies for host, content is the same) https://www.google.com/recaptcha/api/challenge?k=6LcCCsYSAAAAACm9eF4n2ttYMU4TFbDMXMO-Bw2q


6. Which then directs to https://www.google.com/recaptcha/api/js/recaptcha.js that contains an in-body script:
document.write('<script>Recaptcha.widget = Recaptcha.$("recaptcha_widget_div"); Recaptcha.challenge_callback();<\/script>');

SOLVED!!!: click to see commit 2 break throughs, 1 question:

BT1: change into amo register's custom RecaptchaOptions to avoid in-body script.
BT2: have to allow setInterval like 'CSP_OPTIONS = ("eval-script",)'.
Q1: How come amo register does not have "setInterval blocked by CSP" problem even without CSP_OPTIONS?

BT1: In-body script is skipped with a custom RecaptchaOptions
as seen in Google recaptcha's js, note the javascript comma:
if (RecaptchaOptions.theme == "custom") {
    if (RecaptchaOptions.custom_theme_widget) Recaptcha.widget = Recaptcha.$(RecaptchaOptions.custom_theme_widget);
    Recaptcha.challenge_callback()
} else 
    document.write('<div id="recaptcha_widget_div" style="display:none"></div>'),
    document.write('<script>Recaptcha.widget = Recaptcha.$("recaptcha_widget_div"); Recaptcha.challenge_callback();<\/script>');
So the entire "else", which contains the in-body javascript, is skipped!

BT2: Make CSP policy allow setInterval
add CSP_OPTIONS = ("eval-script",) into settings.py
solves the "call to setInterval blocked by CSP" issue (seen in Firebug).

Q1: Why doesn't amo have this issue?

Solved: because amo has CSP_REPORT_ONLY, meaning that CSP is not actually enforced, but only reported!

---------

Have to get around setInterval(). CSP only blocks setInterval if it's called with a string argument.

So let's call it with a function!

Continued on this post.

How Add-ons Mozilla does ReCaptcha

Firefox add-ons register

Code comments are potentially mine.

1. def register in apps/users/views.py
@anonymous_csrf
def register(request):
    if request.user.is_authenticated():
        messages.info(request, _("You are already logged in to an account."))
        form = None
    elif request.method == 'POST':

        form = forms.UserRegisterForm(request.POST) # Always have recaptcha

        if form.is_valid(): # is_valid() does all the form clean()
            ... [save form stuff] ...
    else:
        form = forms.UserRegisterForm()
    return jingo.render(request, 'users/register.html', {'form': form, })

2. UserRegisterForm has ReCaptchaField. File: apps/users/forms.py
import captcha.fields

class UserRegisterForm(happyforms.ModelForm, PasswordMixin):
    passwords ...  
               
    recaptcha = captcha.fields.ReCaptchaField()

    ... irrelevent stuff ...

3. captcha.fields.ReCaptchaField() in zamboni/vendors (not shown on zamboni github), but it's on Mozilla's django-recaptcha
from django.conf import settings
from django import forms
from django.utils.encoding import smart_unicode
from django.utils.translation import ugettext_lazy as _

from recaptcha.client import captcha

from captcha.widgets import ReCaptcha

class ReCaptchaField(forms.CharField):

    default_error_messages = {
        'captcha_invalid': _(u'Invalid captcha')
    }

    def __init__(self, *args, **kwargs):
        self.widget = ReCaptcha
        self.required = True
        super(ReCaptchaField, self).__init__(*args, **kwargs)

    def clean(self, values):
        super(ReCaptchaField, self).clean(values[1])
        recaptcha_challenge_value = smart_unicode(values[0])
        recaptcha_response_value = smart_unicode(values[1])
        check_captcha = captcha.submit(recaptcha_challenge_value,
            recaptcha_response_value, settings.RECAPTCHA_PRIVATE_KEY, {})
        if not check_captcha.is_valid:
            raise forms.util.ValidationError(
                    self.error_messages['captcha_invalid'])
        return values[0]

Which, btw is exactly what I have for my ReCaptchaField. The ReCaptcha widget will ultimately introduce an in-body javascript.
Click to read more about ReCaptcha and In-Line Javascript / CSP

So the only difference in reCaptcha is how it's displayed on the html page. Let's investigate.

4. Register page template: apps/users/templates/users/register.html, taken from step 1 views.py
{% block js %}{% include("amo/recaptcha_js.html") %}{% endblock %}
...
{% if settings.RECAPTCHA_PRIVATE_KEY %}
    {{ recaptcha(form) }}
{% else %}
    <p>
       Welcome Robots, ReCaptcha has been disabled for your convenience.
       Spam at Wil.
     </p>
{% endif %}
The apps/amo/templates/amo/recaptcha_js.html has:
{% if request.user.is_anonymous() %}
  <script type="text/javascript" src="{{ settings.RECAPTCHA_URL }}"></script>
{% endif %}
where
# in settings.py
RECAPTCHA_PUBLIC_KEY = "blah"
RECAPTCHA_PRIVATE_KEY = "blah"
RECAPTCHA_URL = ('https://www.google.com/recaptcha/api/challenge?k=%s' %
                 RECAPTCHA_PUBLIC_KEY)

Unless you have the private key (which bots don't), you can see the recaptcha form.


5. def recaptcha() in apps/amo/helpers.py
Read about the inclusion_tag
@register.inclusion_tag('amo/recaptcha.html')
@jinja2.contextfunction
def recaptcha(context, form):
    d = dict(context.items())
    d.update(form=form)
    return d

6. recaptcha.html lives in apps/amo/templates/amo/recaptcha.hhtml"
{% from 'includes/forms.html' import required %}
<label for="recaptcha_response_field">
  {{ _('Are you human?') }} {{ required() }}
</label>
{% trans %}
  <p>
    Please enter <strong>both words</strong> below,
    <strong>separated by a space</strong>.
  </p>
  <p>
    If this is hard to read, you can
    <a href="#" id="recaptcha_different">try different words</a> or
    <a href="#" id="recaptcha_audio">listen to something</a> instead.
  </p>
{% endtrans %}
<div id="recaptcha_image"></div>
<p>
  <input type="text" name="recaptcha_response_field"
         id="recaptcha_response_field" size="30" />
</p>
<p><a href="#" id="recaptcha_help">{{ _("What's this?") }}</a></p>
{{ form.recaptcha.errors }}

7. div ids link to function in javascript here: media/js/zamboni/users.js
// Recaptcha
var RecaptchaOptions = { theme : 'custom' };

$('#recaptcha_different').click(function(e) {
    e.preventDefault();
    Recaptcha.reload();
});

$('#recaptcha_audio').click(function(e) {
    e.preventDefault();
    Recaptcha.switch_type('audio');
});

$('#recaptcha_help').click(function(e) {
    e.preventDefault();
    Recaptcha.showhelp();
});
These Recaptcha's functions are defined in Google's recaptcha.

Saturday, July 9, 2011

Find where a module is

Run python manage.py shell.
Import captcha.fields
help(captcha.fields) or captcha.__file__ or captcha.fields shown below
    In [3]: import captcha.fields
     
    In [4]: captcha.fields?
    Type:      module
    Base Class:     <type 'module'>
    String Form:    <module 'captcha.fields' from '/Users/jbalogh/dev/zamboni/vendor/src/django-recaptcha/captcha/fields.pyc'>
    Namespace:      Interactive
    File:      /Users/uname/dev/zamboni/vendor/src/django-recaptcha/captcha/fields.py
    Docstring:
        <no docstring> 

It's actually here: django-recaptcha already done before!!, not here. I wish I had know that the django-recaptcha existed before I did this.

Tuesday, July 5, 2011

ReCaptcha on Django

These steps are a simplified/less-hassle version of marcofucci's directions. Thank you marcofucci for helping me setting up recaptcha initially.

1. Get your public and private recaptcha keys

2. Add keys to your settings: settings_local.py, or settings.py if you don't have your code in public
        RECAPTCHA_PUBLIC_KEY = 'insert your public key'
        RECAPTCHA_PRIVATE_KEY = 'insert your private key'

3. Download Python's Recaptcha-client
        1. cd to the directory, e.g. recaptcha-client-x.x.x
        2. python setup.py install

Click for alternative steps 3 and 4
3. pip or easy_install to get Python's Recaptcha-client

4. don't copy anything

5. all the same except the line
from your_app.captcha import submit, displayhtml
is changed into
from captcha import submit, displayhtml

4. Copy captcha.py to your app folder: cp recaptcha-client-x.x.x/recaptcha/client/captcha.py path_to_your_django/apps/your_app So now you should see that this exists: path_to_your_django/apps/your_app/captcha.py

5. Add ReCaptchaField into your forms Differences to marcofucci's step 4 are highlighted.
#add to path_to_your_django/apps/your_app/forms.py

from django.conf import settings
from django import forms
from django.utils.encoding import smart_unicode
from django.utils.translation import ugettext_lazy as _
from django.utils.safestring import mark_safe

from your_app.captcha import submit, displayhtml

class ReCaptchaField(forms.CharField):
    default_error_messages = {
        'captcha_invalid': _(u'Invalid captcha')
    }

    def __init__(self, *args, **kwargs):
        self.widget = ReCaptcha
        self.required = True
        super(ReCaptchaField, self).__init__(*args, **kwargs)

    def clean(self, values):
        super(ReCaptchaField, self).clean(values[1])
        recaptcha_challenge_value = smart_unicode(values[0])
        recaptcha_response_value = smart_unicode(values[1])
        check_captcha = submit(recaptcha_challenge_value,
            recaptcha_response_value, settings.RECAPTCHA_PRIVATE_KEY, {})
        if not check_captcha.is_valid:
            raise forms.util.ValidationError(self.error_messages['captcha_invalid'])
        return values[0]

class ReCaptcha(forms.widgets.Widget):
    recaptcha_challenge_name = 'recaptcha_challenge_field'
    recaptcha_response_name = 'recaptcha_response_field'

    def render(self, name, value, attrs=None):
        return mark_safe(u'%s' % displayhtml(settings.RECAPTCHA_PUBLIC_KEY))

    def value_from_datadict(self, data, files, name):
        return [data.get(self.recaptcha_challenge_name, None),
            data.get(self.recaptcha_response_name, None)]

6. Put it in my form! In contrast to, marcifucci's version, I use the built-in forms.
#add to path_to_your_django/apps/your_app/forms.py

from django.contrib.auth import forms as auth_forms

class UserCreationForm(auth_forms.UserCreationForm):
    recaptcha = ReCaptchaField(label="I'm a human")

class AuthenticationForm(auth_forms.AuthenticationForm):
    recaptcha = ReCaptchaField(label="I'm a human")

7. See my examples on github:
- step 4 copied in captcha.py
- step 5, 6 forms.py

8. If you have more questions, feel free to email me.

Wednesday, June 29, 2011

Getting reCaptcha for django

How I got it to work, in another blog

Things that didn't work:
http://seeknuance.com/2008/03/18/integrating-recaptcha-with-django/

(I tried the other option of http://stackoverflow.com/questions/2275806/easy-to-use-django-captcha-or-registration-app-with-captcha/2275996#2275996  and installed django-registration, but I get a error: "ImportError at /msw/register No module named backends.default")


recaptcha client installation output
recaptcha-client-1.0.6$ python setup.py install
running install
running bdist_egg
running egg_info
writing requirements to recaptcha_client.egg-info/requires.txt
writing recaptcha_client.egg-info/PKG-INFO
writing namespace_packages to recaptcha_client.egg-info/namespace_packages.txt
writing top-level names to recaptcha_client.egg-info/top_level.txt
writing dependency_links to recaptcha_client.egg-info/dependency_links.txt
writing requirements to recaptcha_client.egg-info/requires.txt
writing recaptcha_client.egg-info/PKG-INFO
writing namespace_packages to recaptcha_client.egg-info/namespace_packages.txt
writing top-level names to recaptcha_client.egg-info/top_level.txt
writing dependency_links to recaptcha_client.egg-info/dependency_links.txt
reading manifest file 'recaptcha_client.egg-info/SOURCES.txt'
writing manifest file 'recaptcha_client.egg-info/SOURCES.txt'
installing library code to build/bdist.macosx-10.4-x86_64/egg
running install_lib
running build_py
creating build
creating build/lib
creating build/lib/recaptcha
copying recaptcha/__init__.py -> build/lib/recaptcha
creating build/lib/recaptcha/client
copying recaptcha/client/__init__.py -> build/lib/recaptcha/client
copying recaptcha/client/captcha.py -> build/lib/recaptcha/client
copying recaptcha/client/mailhide.py -> build/lib/recaptcha/client
creating build/bdist.macosx-10.4-x86_64
creating build/bdist.macosx-10.4-x86_64/egg
creating build/bdist.macosx-10.4-x86_64/egg/recaptcha
copying build/lib/recaptcha/__init__.py -> build/bdist.macosx-10.4-x86_64/egg/recaptcha
creating build/bdist.macosx-10.4-x86_64/egg/recaptcha/client
copying build/lib/recaptcha/client/__init__.py -> build/bdist.macosx-10.4-x86_64/egg/recaptcha/client
copying build/lib/recaptcha/client/captcha.py -> build/bdist.macosx-10.4-x86_64/egg/recaptcha/client
copying build/lib/recaptcha/client/mailhide.py -> build/bdist.macosx-10.4-x86_64/egg/recaptcha/client
byte-compiling build/bdist.macosx-10.4-x86_64/egg/recaptcha/__init__.py to __init__.pyc
byte-compiling build/bdist.macosx-10.4-x86_64/egg/recaptcha/client/__init__.py to __init__.pyc
byte-compiling build/bdist.macosx-10.4-x86_64/egg/recaptcha/client/captcha.py to captcha.pyc
byte-compiling build/bdist.macosx-10.4-x86_64/egg/recaptcha/client/mailhide.py to mailhide.pyc
creating build/bdist.macosx-10.4-x86_64/egg/EGG-INFO
copying recaptcha_client.egg-info/PKG-INFO -> build/bdist.macosx-10.4-x86_64/egg/EGG-INFO
copying recaptcha_client.egg-info/SOURCES.txt -> build/bdist.macosx-10.4-x86_64/egg/EGG-INFO
copying recaptcha_client.egg-info/dependency_links.txt -> build/bdist.macosx-10.4-x86_64/egg/EGG-INFO
copying recaptcha_client.egg-info/namespace_packages.txt -> build/bdist.macosx-10.4-x86_64/egg/EGG-INFO
copying recaptcha_client.egg-info/requires.txt -> build/bdist.macosx-10.4-x86_64/egg/EGG-INFO
copying recaptcha_client.egg-info/top_level.txt -> build/bdist.macosx-10.4-x86_64/egg/EGG-INFO
zip_safe flag not set; analyzing archive contents...
creating dist
creating 'dist/recaptcha_client-1.0.6-py2.7.egg' and adding 'build/bdist.macosx-10.4-x86_64/egg' to it
removing 'build/bdist.macosx-10.4-x86_64/egg' (and everything under it)
Processing recaptcha_client-1.0.6-py2.7.egg
Copying recaptcha_client-1.0.6-py2.7.egg to /Users/haoqili/.virtualenvs/playdoh/lib/python2.7/site-packages
Adding recaptcha-client 1.0.6 to easy-install.pth file

Installed /Users/haoqili/.virtualenvs/playdoh/lib/python2.7/site-packages/recaptcha_client-1.0.6-py2.7.egg
Processing dependencies for recaptcha-client==1.0.6
Finished processing dependencies for recaptcha-client==1.0.6