WebSource - a web based software development environment
By azul in Internet
Tue Feb 13, 2001 at 08:22:23 PM EST
Tags: Software (all tags)
Here I introduce an interesting idea I've been thinking about in the last few days. This is a web-based software development environment, aiming to substitute your text editor, CVS, bug-tracking systems and other tools, fully integrating them. You could, in the same window, see a function's code, its assertions (separated from the code), good documentation about the way it works, links to other relevant parts of the program, a list of bugs linked to the function, a set of messages posted to the function, links to all the earlier versions of the function (along with information about their author and the dates on which they were written) and more. WebSource also borrows many ideas from literate programming and uses them exhaustively. Even though WebSource is not even vaporware, I'm posting this to Kuro5hin hoping to generate some discussion about its design that make help me in the future during implementation (in case this needs to be mentioned, if I implement it I'll release it under a free license, probably the GPL).
I have this idea that I decided I would write down and let others review
for some time before I actually begin to implement it (if I do). I must state
that I'm not sure this program is actually worth implementing, but I find the
Ladies and gentlemen, I give to you WebSource.
Okay, only its requirements.
WebSource is an innovating interface for developing software. In this
paper I will review some of the features it should have once ready. I will
also suggest some other interesting ideas that I have and ask for your
Why is it so innovating? Lets make a quick list of its features, that
I'll explain with more details bellow.
- Web-based interface
I know some of you might get a "YUCK that would suck" reaction. Just
wait, let me explain all the benefits of a web-based interface. But yes, you
edit the source code using HTML forms.
- Versions Control / Central Repository
Okay, this on it self can't be called innovating. Its integration with
the other features certainly is. No need for CVS if you use
- Bug Tracking System
Some of the advantages of the web interface: it allows us to perform bug
tracking efficiently. It would be simple for programs to automatically report
bugs to the central database (more on this soon).
- Literate Programming
It incorporates most of the ideas common in tools for literate
programming. Basically, it tries to add information to the code. This
information is, however, not limited to documentation but includes many other
things that we'll see soon. Also, different than most current literate
programming approaches, it will not be designed to produce printed
documentation but to produce it in an efficient hypertext system.
To be exact, WebSource is an application that runs on the web server (lets
say via Common Gateway Interface; in reality it'll be something more efficient
such as Apache Modules). It lets developers work on a given program, using
their web browser as the interface used to edit its source code. It will
maintain the code in the system in the web server.
This system is designed specially for developing free software. Though
proprietary software might benefit as well by its use, we are focusing on the
typical development of large free programs.
Using WebSource, we view a program as a group of many small interrelated
programs. They don't have to be functions, just small groups of code that do
something. We will refer to them as sections. If you don't understand the
meaning of sections, refer to standard documentation for literate
When developing software using most of the common literate programming
tools, you end up writing a whole source file where it is impossible to see the
logic structure of your programs efficiently. You have to actually produce the
documentation (a process known as weaving) before being able to see things such
as the relations between the different sections of your program. We feel that
this information empowers the developer and should be available at all stages
of the development.
Basically, your whole program is the big section, and its broken down into
smaller sections. You have a tree where each section can have as many
sub-sections as you desire. The leaves in the tree represent sections that
have nothing but code written in your programming language of choice; all other
nodes can contain both code and references to other sections (this references
work as inclusions of other sections; think like function calls or like
C-preprocessor macros that are defined as functions).
WebSource will be language independent. It will support as many programming
languages as possible, each independently described in individual configuration
files. Initially, we plan to add support for either C, Ruby or Java,
eventually adding other programming languages as they are demanded by the
Lets now look at the information kept for every section of code kept in
control by WebSource:
- Version Control
As the program evolves, the different stages of every section will be
kept. For every different version of a section, we will keep the author's
comments (stating the reasons for the change) and the date it came into effect.
It should be possible to see all the different versions that have been
available for every section.
Unlike other literate programming tools (specially CWEB), we will make
it possible to pass parameters to the sections. This will work pretty much
like the parameters to C-preprocessor macros defined as functions: all the
occurrences of the name of a parameter will be replaced with whatever the
caller section passes. This is related to the assertions, defined
- Source Code
Here goes the actual source code for the section, along with a mechanism
for inclusion of other sections inside the current section's code.
- Special comments by the author
The author should state what the code does and why she choose to write
it the way she did, along with any useful information.
- Forum for discussion
We feel that by having a forum where everyone can post notes, we will
encourage people to discuss about the different features and bugs in every
section. This should make it easy, for example, to suggest different
alternative algorithms. The forum should be a little smarter that most common
message boards: for example, it should make it possible to mark a given message
as a bug report (with different priorities for each bug report). Most forums
would be empty, but others might generate a lot of discussion.
- Links to other sections
When viewing a section, it should be possible to see other sections that
(a) the current section depends on as well as those that (b) depend on the
current. This should make it very easy to navigate the source code.
- Assertions (preconditions and postconditions for every section)
We will provide a special field for preconditions and another for postconditions. This takes the form of source code. They could be included in the "Source Code", but we feel it's appropriate to separate them so depending on the type of build the program can be compiled with or without them.
Keeping them separate from the actual source code might make it easier to
find Heisenbugs: side effects of the assertion checks that cause bugs when they
are not included.
Finally, a third advantage of keeping assertions separated is that this
facilitates programming by contract.
The system for the assertions must be such that whenever an assertion fails,
the whole ``stack'' is reported (here, the ``stack'' is defined by the
sections, not actual function calls).
The following features are highly desirable in WebSource. Some would be
easier to implement than others. Some of them we are not sure deserve
implementation, but we still find them interesting enough for inclusion.
- Syntax Highlight
It would be desirable for WebSource to be able to display the code
performing syntax highlighting. As the actual highlighting of a section
depends on the part of the program where it was found and this information
might not always be available for a given section, this might turn out hard to
do reliably: WebSource would have to check the places where a given section is
used to perform the highlight. Should this feature be implemented, we feel
that an already existing mechanism for performing syntax highlighting is
reused. Vim's syntax description files come
- Error Detection
WebSource should automatically make checks on the code submitted to see
whether it causes errors. This information should be related to the versions'
- Automatic Indenting
For programming languages that don't care about the way a program is
indented (almost all, but Python, for example, depends on the way the code is
indented), every programmer should be able to state her preferred indentation
style. WebSource would automatically adjust the code for her
- Automatic Building capabilities
WebSource must be able to automatically build an executable from the
source at any given time (assuming, of course, that the code does compile).
It must be easy to download either (a) the resulting source code when all
the sections are integrated (this process is usually known as tangling) or (b)
a binary program (ideally for whatever platform the user chooses).
It must be possible to download this results compressed using different
- Unit Testing
WebSource should allow developers to add different checks on the
different functionality provided by the program. We borrow this idea from
Extreme Programming because we think it works very well. Whenever a new
version for a section is submitted, all the unit tests will be run and the
results added to the versions information.
For very large programs, it will be possible to disable this feature and
have the checks be performed periodically (every night at 4 AM, or
It must be easy to know the results of all the tests at any given
- Object Oriented Programming (OOP)
This is a very popular programming methodology that we want to support
as well as we can. We believe that the system as is defined blends very well
with OOP, but we will listen to any suggestions that might improve it.
- Use of XML to provide interfaces to other programs
It should be possible to export all the information in XML to make it
easier for agents other than a browser to interface with WebSource. That way,
anyone could design agents other than the web browsers and access all of
WebSource's functionality through them.
- Use of HTML to display and edit code
Some might see this not as a feature but as a necessary problem of our
system. However, this might actually make a very nice editor. Consider,
first, that most of the sections will never be bigger than a few lines. Also,
consider that we can make hypertext links when displaying code from things such
as variable names, types, functions and so on. Just think about all the links
that the following C code could have:
for (i = arg_head; i != NULL; i ++)
if (!mytype_equal(i->val, current))
We can think of around 6 different links. And no, the code wouldn't look
all in blue and underlined, we would use good HTML and CSS to ensure that the
links are not intrusive and don't make the code hard to read.
- Batch Edition
Okay, so you are just starting and want to create 200 sections with
code, and you'd rather do it with your favorite text editor than with poor
browsers' textareas? Sure, it should be possible to do that and easily
integrate the resulting text file with WebSource (say use the browsers Upload
button to update the program: WebSource then checks the file uploaded and, if
it doesn't have any errors, it integrates the changes).
To make this easier, it should be possible to export-out sections: you
should be able to check any given number of sections and your browser would
download a file with them all that you can edit using your favorite text editor
to update them all at once.
- Locking Of Sections
This is required to allow many persons to work on a given program at the
Locking should be used at many parts of the WebSources system.
- Bug Reporting Mechanism
WebSource must come with simple interfaces that allow users with no
programming knowledge to report bugs. The bugs can be eventually linked to
sections by the developers. The standard information for every bug report is
kept: platforms affected, priority, status, reporter, date when it was reported
(and all the dates when its information changes) and so on.
Another idea here, specially related with the assertions information for
every section, is to have WebSource build programs in a way such that whenever
an assertion fails, the program automatically reports it to the server using
HTTP. As the actual implementation for this depends on the programming
language as well as the kind of program, this must be implemented for every
program (though we believe code reuse will certainly become a pattern here) as
a section that gets executed whenever an assertion fails.
- Stats about the Development
It should be possible to export statistics about the development of the
program such as the total number of lines written by each programmer or the
average number of lines per unit of time (here the time would be the time
during which a section is locked).
- Graphical Information on the Program
WebSource should be able to render graphs with useful information about
the program. This includes the following:
- Graph about sections usage, so one can know which are the most used
sections. And which sections are not used anywhere. Stuff like that.
- Tree view for all sections. It should be possible to view all the sections
as a tree. The user should be able to specify the desired depth of the
- Results of Unit Tests over time. This could be used by some as a way to
measure the progress of the project.
- Number of bug reports as a function of time. This would make a lot of
sense if the automatic code for reporting assertion fails is used.
- Language dependent graphics. For example, for OOP, it should be possible
to see graphs for the different classes (such as the sections used by each
method of a given class).
Feel free to suggest others.
- Email Interface
This would allow the users to receive information on the status of the
program periodically at their email. For example, reports on the bugs or the
changes to the sections. It should be possible for to reply to those mails and
have WebSource automatically incorporate the content or forward it to the
- User Authentication
WebSource needs a good system of accounts giving each user different
privileges. It would be interesting to make it so a given user can only edit
certain sections. Also, this system should keep settings such as the preferred
indentation-style for every user or their email preferences.
- Mozilla and advanced features
other persons might disagree and argue that some special features of Mozilla,
not available in other older browsers, should be used. I think WebSource
should be designed so none of its significant features is missing in a
Using WebSource would not be a win-win situation, but more like a sort of
tradeoff. Specifically, writing new code might become slower as you are forced
to add information to it. However, it might make it easier to anyone
interested in modifying the software to understand it, find the relevant code
to implement his modifications and find out if his changes break any other
sections of the program. This should, in the long term, make software more
Anyway, that's pretty much all the ideas I have for now, and it should give
you a clear view of what WebSource will be (assuming it eventually is). I
welcome any comments and criticism at my usual email address, email@example.com.
I will not work on WebSource for some time, hopefully giving you enough time
to contact me and send me all your suggestions and criticism.