Tuesday, April 1, 1997

ClariNet News and Time

ClariNet was the first commerical content provider on the Internet, starting way back in 1989. It took wire service news (at various times, from the AP, UPI, Agence France-Presse, and Reuters), classified them into Usenet newsgroups, and allowed sites to subscribe to those newsgroups for a fee.

For a considerable time I was primarily responsible for article classification. I had written a proposal suggesting a finer-grained classification system than the newsgroups allowed, which was not accepted by ClariNet's management. At one point I realized I was being a bit overzealous in advocating my proposal. This was my attempt to poke fun at myself.

ClariNet News and Time

By Aaron Priven, 1 April 1997

Time, Usenet, and Newspapers

ClariNet's news has been, and is still, chiefly delivered in Usenet format. One of the features of Usenet is that it is divided by subject, and not by time. (Most other conferencing systems work this way as well.) The first story on January 1 directly follows the last story from December 31, with nothing to mark the change. Indeed, it can be that stories on different dates are mixed in together, due to embargoes and the delays caused by editing.

Newspapers, and even most online news services, do not work this way. A newspaper is primarily classified by date. Today's newspaper contains today's news, and the strongest of divisions separates it from yesterday's news.

It is possible that since most people are used to having today's news separate from yesterday's, that ClariNet news is counterintuitive and less usable for that reason. It may be desirable to separate news by day, rather than by subject.


Possibly, this could be done by modifying the web extraction program, providing clear delimitation between days in only that form of the news. However, that plan is for another day than this. Here I will discuss potential implementation proposals within the Usenet form.

To implement this within Usenet, it will be necessary to create new newsgroups associated with the days. There are two basic questions to be answered. The first is whether one chooses relative or absolute dating. The second is whether one chooses to crosspost the news to a new set of date-specific groups, or to incorporate the date into the existing group names themselves. These questions will need to be answered before an implementation plan is produced.

Relative or absolute dating. With relative dating, one creates groups such as clari.news.today, clari.news.yesterday, and the clari.news.days-ago hierarchy (c.n.days-ago.two, c.n.days-ago.three, and so on, up to c.n.days-ago.twenty-one). With absolute dating, one creates a separate group for each date: clari.news.y1997.april.d1, clari.news.y1997.april.d2, and so on. (Unfortunately Usenet group components cannot consist solely of numbers.)

Each has its advantages and disadvantages. Relative dating requires no regular change in group names. The groups themselves would stay the same from day to day. However, the news in those groups must change regularly. This would require canceling every article at midnight every day and sending out a replacement in the previous day's group. (So that, for example, at midnight all the contents of clari.news.today would be canceled and reissued in clari.news.yesterday.) One unanswered question is which midnight to use. Most of our customers are North American, so midnight ET might be most appropriate. However, as we expand into world markets it may be necessary to use midnight GMT. Or perhaps it will be necessary to have several editions, each with a separate midnight: European time, Japanese time, and so on. The mechanism for doing this last would be difficult.

Absolute dating does not require the repeated cancelation and reissuance of articles. It does, however, require the issuance of new newsgroups each day. This would require more cooperation on the part of news administrators than we have had previously. It would, of course, be possible to specify the newsgroup names ahead of time, so that an administrator would only have to create the new groups once every few months. This would result in a lot of empty groups; however, unlike some of our currently empty groups, this would at least be intuitive (it should not be surprising that a group dated three weeks from now should be empty). It would still be preferable to have the groups created automatically each day, if news administrators will allow it.

Crossposting or incorporation into current groups. Essentially, this is the question whether these new date groups are to be an adjunct to the current system, or are to be incorporated into the existing system itself. Crossposting would imply that all news stays in the current group set, but is simply added to a new set of groups chosen by date. This would be relatively simple, but would not yield results for most of the readers, who will certainly not want only to have date-oriented selection. (Even newspapers have sections for particular kinds of stories.)

The other alternative is to create a separate hierarchy for each day: clari.today.biz.briefs, clari.today.biz.earnings, and so on, up to clari.days-ago.twenty-one.world.terrorism and clari.days-ago.twenty-one.world.top. This would provide the maximum benefit for those readers who want to find news by date as well as by subject. However, this would require a substantial number of new groups. For the relative dating system, it would be necessary to create 9,780 new groups, plus rename the current 489 groups to include the "today" marker. For the absolute system, it would be necessary to create 489 new groups each day (although it may be possible to only send out newgroup messages for those groups that are actually used; this could mean a significant reduction on weekends). This is not a minor undertaking.

It may be possible to have some compromise between the two sets. For example, there could be only a few categories marked with dates, perhaps something like the one-star set: clari.today.biz, clari.today.usa, and so on. However, these are still likely to have great volume. Creating a set with reasonable volume is likely to be rather detailed; not so much so as the full set, but close to it.


These questions of which groups to use must be answered before any implementation plans begin, as the implications are serious. However, attracting readership is very important, and we must do what we can to do this. This may be a first step.