simple matter of programming?

[cellio]

Maybe the process of writing this down will help me clarify the problem in my head.

We have a mucking huge API for our Java toolkit. (Well, incipient toolkit.) My goal is to make a smaller API. There's a lot of stuff that's currently public that doesn't need to be and cutting that out is easy. The hard part is making sure that the new, smaller API does not refer to classes or interfaces that are no longer public. (By "refer to", I mean appearances in signatures. For example, if the constructor of PublicThingy requires that you pass a NoLongerPublicThingy, we're in trouble.)

There has *got* to be a programmatic solution to this problem -- either out there already or feasible to write. What I want to be able to do is specify an API, crank it through some process, and get back either "ok" or "oops, still have refs to the following: ...".

This is an API, not an actual smaller code base, so it's not as simple as removing files and running the Java compiler. Everything will still be there; it's a question of what we expose.

The specification of the API is the javadoc. I of course have access to the source code, but I don't think crawling through that is the way to go.

Some args to methods will be objects in our API; others will be Java classes (JDK) or base types (like int). So I can't just compare signatures to our list of classes; I need a way to weed out the standard Java stuff. So I need to be able to tell the difference between NoLongerPublicThingy and Double.

One way to attack the problem would be to generate the lists of permissible objects and then run all the signatures through something that compares to that list. Ugly and slow, but I suppose it would work. Feels like a gross hack, though. (Javadoc generates the lists as part of its normal processing, so I can just grab the list from the JDK Javadoc.)

A more elegant solution would involve modifying Javadoc processing itself to do this checking as it goes. After all, it's already doing some processing on each signature to do its formatting and links and stuff, so why rewrite the code that parses signatures? On the other hand, I don't know how feasible it is to modify Javadoc. (Yeah, it's *possible*, but I don't know anyone who has actually *done* it.)

Another approach is to treat this as a general interface-validation problem and assume that someone out there has relevant tools, maybe built into some sort of development environment. Sounds expensive and maybe hard to find, though.

I was wrong; writing this didn't help me sort it out. Unless this entry prompts one of my readers to say "You idiot! Why don't you just <answer>?".

Comments

[dr4b]

I've modified javadoc. It's disgusting.

You can even get the source files for the standard doclet from here and see some of their API documentation here. They have some tips on their Javadoc FAQ for how to write your own doclets. The main thing is that, in order to change the output at all, you pretty much need to subclass the standard doclet, and they *really* didn't seem to intend that to be done, by the way the doclet API is designed. To make your own customized javadoc thingies isn't quite so hard, like if you want to add your own tags and output them in a separate file (we do something like that for keeping a page of all our classifiers and extractors), or if you want to just go through and output all the class names, etc... you can write a doclet and generate desired output separate from the normal javadoc build. But, it's yucky. I don't think it would help this problem, either. I didn't delve into the part where it parses the signatures all that closely, but it looked awful.

[cellio]

Yeah, most of my hesitation around a javadoc-based solution came from remembering how much trouble you were having with it...

[dvarin.livejournal.com]

This sounds eerily like the task I had to do at Clairvoyance when we split the tree there, except more complex because you can't rely on #include lines.
I'm going to venture that the perl script (or whatever you use to compare) may be much more aesthetically displeasing, but it may be easier to write more quickly than a doclet-based solution.
On the other hand, as I found out, if you write the perl script you will also need to maintain it until someone gets annoyed enough to rewrite it in Make.

[ralphmelton.livejournal.com]

Carl is right that we did something like this at Clairvoyance.

For the validation part of that that you're asking about, I wrote some scripts with Discover to check for public things that referred to non-public stuff. (Sadly, there was so much of that that we couldn't fix it all; we just had to accept that in places.)

Of course, saying that we used Discover doesn't particularly help you...

[cellio]

Yeah, I miss having Discover available. It was a very handy tool. Not the sort of thing that we're likely to be able to afford even if they support Java (and I don't know if they do yet), but I can dream. :-)

[ralphmelton.livejournal.com]

It was kind of handy, but we used only a fraction of its power, and I'm not convinced that we got enough benefit to match the cost.

[cellio]

I agree that we didn't get our money's worth out of it, but a fair bit of that is due to when and how we used it. Basically, by the time we got it we weren't willing to make the sorts of architectural changes that using Discover would have uncovered; therefore we only used it for smaller tasks that didn't warrant the price tag. If we'd had something like that back when DPF was being developed I suspect the results would have been effective enough that folks would have *had* to switch over to DPFMT. But it's the sort of thing that supports active development better than it supports maintenance. Or so it appeared to me, anyway. (It's not like we ever really got to talk to live users, as opposed to their support people.)

[seanmcguire.livejournal.com]

Are you making classes sthat aren't actually in the API non-public? If so, with Java introspection (or maybe they call it reflection) it should be programatically possible to look at every method in a class or set of classes and ensure that all of its arguments are public classes. If I still had my Java chops (I've been programming in other languages for about 7 months now so they're a bit rusty) I'd say "Oh, I can whip that up for you in fifteen minutes." In fact it would take me several hours, but I'd think it'd take me fifteen minutes. :)

[cellio]

Unfortunately, it's not going to be the case that all public classes will be part of the API. While we can't prevent programmers from using undocumented classes if they somehow discover them, of course, we can say that we aren't going to support them in that. :-) (The developers assure me that there are good reasons these classes need to be public -- mostly having to do with calls that need to originate in other packages.)