Publishing, Search, Fulfilment and Conversation as four pillars of any software system …

I am sure it has happened to you as well : After thinking thru & working feverishly on a problem and having convinced that one has THE solution, suddenly one come across an insightful discussion that totally changes one’s point-of-view !

It has happened to me occasionally and had one occasion today when I read JP’s insightful blog “Facebook and the Enterprise: Part 4: Four Pillars”

In the name of brevity let me get to the point with a brief executive background …

What is a good story without a little rambling eh ? ;o)

I am part of a team at the San Jose Education Foundation working towards creating a destination for teachers – local schools first and then, of course, world domination ;o)

We were following three vectors – roughly speaking -

  • SIMS(Student Info management System),
  • Sharing Teacher Innovations &
  • Teacher tools.

Of these, after talking with the teachers and working with the board, we zeroed in on the domain of lesson plans and teacher tools around it.What we need is a destination for teachers, which would offer value for their precious time – so LessonOPOLY is born ! BTW, our fearless Director Of Innovation Gina came up with that name !

Incidentally this is a poster child for Web 2.0 because, we want a derivative based [1] collaborative creation of lesson plans as well as some form of social networking/interactivity.

We have this romantic notion of two classes from different geographies taking a field trip (to the same place, at the same time, by design not by accident) based on a shared lesson plan and coordinating the trip thru LessonOPOLY so that the students can meet and share their learning experience !

In short, the main features we want to offer to teachers are :

  1. Collaborative creation of lessons,
  2. A derivative based, repurposable, remixable, on-demand interchange multi-media content capability (it sounds more lofty than it actually is ;o))
  3. Ability to meaningfully stitch a fabric (of lessons, in this case) from a vast sea of materials which differ in content grade, content relevance, granularity and format diversity [2]
  4. Ability to perform semantic search on the federated content with contextual attributes meaningful to teachers &
  5. Finally, of course, social networking and other second order artifacts (For example, the above mentioned simultaneous field trip based on a shared lesson plan !)

For #3 and #4, what we realized was that there are lots of content available, many in lesson plan format ! But still it is not easy for a teacher to get to the right one to use for a particular lesson or an objective. And teachers, being overworked they are, would spend their time face to face with students than wading thru 100,000 items on a search page !

Our solution was a semantic search based on technologies like clustering, Kernel Methods and SVMs, with attribute based tagging. I was convinced that this is a search problem, until … I read JP’s blog and saw the words publishing, search and conversation in a coherent/cohesive way that was relevant to our work ! I would possibly add context in the mix as well …

After reading JP, I realized that we are looking at is publishing (based on a derivative based approach [1]), search and conversation ! In fact, the derivative based lesson plan creation is a conversation, not the traditional voice/im/e-mail type, but definitely a conversation or a narrative as mentioned by Dr.Norvig here (at the bottom of the interview)

And as JP mentions, we are building a context based visualization tool to enrich and improve the vast sea of content out there ! This is not a find problem, but an information overload problem ! In fact, we are anti-search ! We want to throw out stuff, not find them !

Yet Brutus is an honorable man and this is still a search semantic overlay for a derivative / tagging based collaborative content creation & aggregation. We still need multi-variate, multi-modal clustering. But my Support Vector Machines, Kernel Methods, Kalman Filters and Clusters should search for (and make probabilistic inferences on) irrelevance (based on context, of course, which would include location, temporal attributes, abstract past interactions and wisdom of the crowds) !

Thanks JP for the insights ! Your observations are true not only for enterprises (i.e. where we work) but also where we play & learn !

P.S : I think this is possible more now than before. See a discussion on semantic technologies …

Update [Oct 2, 2007] : Came across Sramana’s blog which also talks about Context and Vertical Search as pillars of Web 3.0.

Digg This

[1] By derivative, I mean collaborative, revision based creation, like a Wiki but which includes multi media ! Then what we see is not 50 intermixed versions of the same materials, but a systemic organization of work by multiple folks, all trying to create a set of coherent lesson plans for their students. Plus tagging, not only about the utility but also the appropriateness for a particular purpose (for example introductory vs advanced, for CA curriculum vs main curriculum, for 3rd graders vs 6th graders, as a first course vs as a second course, …)

[3] A great discussion on the top down approach for semantic web is at

One thought on “Publishing, Search, Fulfilment and Conversation as four pillars of any software system …

  1. Pingback: Ubuntu 7.10 & 2 X QuadCore Dell PowerEdge SC1430 « My missives

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s