in reply to Re^2: Please review my OSCON presentation
in thread Please review my OSCON presentation

Ah, much clearer now. My problem here was that you didn't say that you were only showing the API for a central but incomplete part of the A/B-test framework, one could say the controller interface of the MVC model. The management of tests and versions, their mapping from name to webpage, the logging (for example log_test_activity()), and all the statistical analysis is (in a strict view) part of the API too.

Just mentioning in that slide that "that is the function that selects which version of a test to show a person on the website" would provide context and make this slide so much easier to understand. That sentence may be obvious to a smal l or large part of your audience but would have helped the others to keep up (at least it would have helped me, 100% of this test group ;-). It is also not the whole truth about the routine, but its possible use in the statistics phase to map users to versions can be explained later

"don't have time to do real exercises": Do you mean test presentations or Moritz suggestion to provide examples? If the latter, you don't need *real* exercises/examples. A hypothetical example can be trivial and senseless, it is a way to ground the theory in the practical.

Look at the slide "Behind the scenes". Apart from the first sentence "This does what it says" (which has absolutely no information value at all) this slide could have been replaced with an example where you provide the same information in the context of your example.

Let's say you have (on a previous slide) given the example of a english to suaheli translation page and you want to test it with versions with green and blue background color. 

----------------------------------
Peters first visit
------------------
Peter visits a page -> web server calls get_or_create_test_version(<pa
+ulsid>,'EtoS',['green','blue']);<p>

New User? [small picture of file cabinet with a arrow pointing to a '?
+'] YES.
[small picture of a dice or coin toss] -> random choice of 'green' or 
+'blue'. Let's say 'green'. ->
[picture of string 'peter has green' with arrow to file cabinet ]. 

$version='green'

[ small picture of a green cube which looks like a webpage ]

----------------------------
Peters next visits
----------------
Peter visits a page --> web server calls get_or_create_test_version(<p
+aulsid>,'EtoS',['green','blue']);

New User? [picture of file cabinet with an arrow to string "peter has 
+green"] NO.

$version = 'green'

[ small picture of a green cube which looks like a webpage ]
------------------------

[download]
Yes, this is trivial (and could be improved a lot). But this gives the same information as the replaced slide and because of the pictures and the concreteness is faster and easier to understand and remember.

I don't think that a picture on every slide is beneficial. This will distract many with questions like 'What has that picture to do with the message of this slide?', especially with long presentations. I would put pictures into introducory, title/heading/opening/closing slides (slides 0 and 47 for example) and easy/low complexity slides (slide 4). On all other slides there should be a picture only if it either:
1) contributes to the information on the page (slide 5 is a positive example), or
2) loosens up the presentation by showing a cartoon, a famous saying or a funny picture that has some relevance to this slide. This cartoon or picture must be the first thing on the slide (because it is eye-catching, everyone will read or look at it first) and you have to give the listeners time to view and possibly laugh about this picture.

Disclaimer ;-): All of this IMHO. And apart from the above I like your presentation and find it interesting, but you wanted to know about the bad not the good parts.

Replies are listed 'Best First'.
Re^4: Please review my OSCON presentation
by tilly (Archbishop) on Jul 17, 2008 at 21:20 UTC
    I was up until 2:30 this morning putting pictures in slides, I'm not taking them out without good reason. Perhaps once I have more sleep I'll reconsider and change my mind, but at the moment they seem to have a purpose to me, if only to convey a particular emotion or remind people of the earlier slides which connect to the current ones.

    I'll definitely rework the behind the scenes slide. I don't think that I'll quite go with your idea, if only because I don't have a store of clip art I could use to draw that picture. But I will try to find a way to make that slide do more to justify its existence.

[reply]

      I think the pictures are perfect. Pictures entertain the more creative side of the brain so that it doesn't start distracting the more logical side from paying attention. All of your pictures tie in well with your topic. By reusing images the creative side of the brain will have something to do, by flooding the watcher with memories of what was going on last time they saw that image. Finally, for those who have given up on paying attention to your material (perhaps it's not relevant, perhaps they just came because a friend was coming, perhaps their stomach hurts), pictures give them a reason to keep looking up.

      I help pjf with his talks all the time. We've found through experience, that fast moving slides with lots of pictures, get a much better reaction than a few pictures here and there.

      If asked, I'd say that one picture every 4-5 slides is not a bad goal to aim for, but it should be a *minimum*. It depends a lot on how fast your slides move. I'd suggest you want one picture every 90 seconds. If you spend 2 minutes per slide, that's one picture per slide. If you spend 6 - 10 seconds per slide (as we tend to do these days) then that's on picture every 9 - 15 slides. After enough talks you get a feel for it though.

      Anyway, I'm sorry I didn't comment sooner, but thanks for putting these up; I found it very interesting and am sad I wasn't as OSCON to attend!

      jarich

[reply]
        I wish you were at OSCON as well. I would have loved to have met you, and you should have received your award in person. Perhaps you'll be at the next OSCON?

        In the talk I was spending about a minute and a half per slide, and I didn't feel unhappy during the presentation with the pictures. The feedback that I've gotten so far on the talk has been positive. However it was so much work to put together that I'm going to be in no hurry to repeat the experience. :-/

[reply]
      IMHO Slide 29 is much better now. I see you expanded that chapter with a code outline (or I forgot that it was there). Looks good, second best solution (best solution would have been... a picture ;-). Be careful not to dwell too long on these slides, because for programmers this might be obvious as soon as they know the purpose of the function, for non-programmers these are hieroglyphes.

      Btw. I asked one of the best lecturers here at the university whether my advice about the pictures was correct and he said: A picture every 4 or 5 slides is enough and the pictures must be directly connected to the slide otherwise the listeners minds will wander off.

      But if you loathe to throw those pictures out now (or you think my advice makes no sense), leave them in. This is then version A of your lecturer-test. Afterwards you can ask some of the listeners if they had trouble concentrating. Next time you give this presentation, show version B with fewer and more relevant pictures. Naturally that is comparing apples to oranges. You have a different audience and there are other factors like time of day and how good you present and the sample is pretty small. But on the third lecture you have an example A/B test you can talk about ;-)

[reply]
        I actually always had that. I just made it several slides.

        I also disagree on which solution is best. I've seen multiple programmers try to implement that function and mess it up. There are a number of subtle requirements you have to get right, and I don't think there is any clearer presentation of how to get them right than to show code. I'm not concerned about the common case being wrong. I'm concerned about the production hook going missing, programmers doing the random assignment as $person_id % 2, and other similar mistakes.

        I'm leaving the pictures in. I've asked a number of people for feedback, and the consensus of most of them is that the current pictures work well. While I agree that they can be a distraction for someone who is getting lost, I have points where people can resynchronize. And the talk has multiple audiences within any organization, I expect people to lose interest when I'm presenting about stuff they are not interested in. I'd like them to remain somewhat entertained, and then I'd like to get their attention when we move on to a section they care about.

[reply]

In Section Meditations