I'm going to try to reply to your post first, then read the other responses.
I'm also going to assume there are some constraints, like what I can reasonably expect test-takers to complete and test-assessors to assess. An ideal "comprehensive" test would probably take each and every person's whole lifetime at minimum and that's just stupid.
But also, we can assume we're not talking like... a test we need to market or have be popular or able to fit in to some specific purpose (work, school, courtroom, etc) or even be completely understood at first by every person who takes (or assesses) it, though some efforts will be made. The INTP of tests, if you will.
As you already mentioned, type of assessment is going to pose some problems for a huge question like "what's this person's philosophical outlook". There's just too many different variables and the bounds of "philosophy" itself are porous and kind of (as far as I know) came to be understood through a historical/cultural process repeated many times, in many different ways, around the world and then smashed together in more recent times. And we all go "yeah that makes sense, I know what philosophy is" but really? A big mess.
What's remembered, who it's attributed to, what philosophy-equivalent things don't get included just because of their odd origins or having some different social "lineage" completely separate from the "usual philosophers", how all that influences our perception of everything... I mean you'd need to make a test on people's opinions on your test itself by the end of it. Which I guess could improve (or, at least, contribute to) the test.
Which is the first of my ideas - assessors as test takers and test takers as assessors. Of themselves, and each other. As in, you can't (really) take the test in full without also assessing other people's test responses, which then feedback into your response by being public to your "profile" or record of contributions, and other people can give an assessment of your assessments (and what it means for your philosophical view).
How to implement this...It's clear I'd like to use the internet/computer program to create a big filter system, you review 1 person's initial responses, and your shit gets sent to 1 other person's review queue they can pick from... you get someone else's response to your initial responses, and you have a chance to respond to that. Never to delete, only to add. If you contradict yourself, you contradict yourself - though if you want you can note it's a change you're aware of vs something you don't comment on (leaving people to speculate).
There are a few different reasons I think of it as a "filter system". It is a filter for your thoughts, putting you through the process of articulating, expressing, and (if you want) defending them or amending them. One assessor is going to be an equal part of your "results" as you are, but many, many assessors? That approaches a clearer picture of you (and society lol but we can't escape) especially if those results are rich feedback, not just someone ticking off the boxes for cateloging you. Though I entirely advocate for us having some box ticking option. And developing a more rigorous "taxonomy" off of that for some laughs (and something useful? It's unclear).
It would also be a filter of interest - do you really want to get to the heart of your philosophy or not? Though I am trying to hold myself back and not demand something like a year plus of engagement before you get results, which just wouldn't work... I will say, ok this will be a bit more like what iNaturalist is to biologists, less like what MBTI tests is to typology. So, more like what (productive) forum discussions of type are to typology??
It's open-ended. There's a result, but it's a result that can change and be refined the more you are able and want to engage, and the more other people are able and want to engage with you.
I'd want entries to be able to be submitted in a wide variety of formats. For one thing, this will best stand a chance at covering the huge variety of philosophical traditions and ways of being, thinking and for a second thing, cool.
So, there would be a selection of templates for uploads... more quiz question types, essay questions to respond to, you can key in a response to any known/listed philosophical work or post your original material. And also, visual art, music/audio recordings of speech, documents with hypertext, more, possibly.
This is something I want, but accept would be extremely difficult to implement, particularly at a large scale where everyone can interact with everyone and not just 1 philosophy class registering together and only able to see and respond to each other, or something. To host it would be difficult, and to keep it from being used for all sorts of shit not philosophy related, too, would be a stretch. Who would keep it running? But maybe it could work. Forums work, vaguely speaking. Communities dedicated to all sorts of scientific data work, again, generally.
Cutting it down to the bare essentials, I'd keep a mix of quizzes (multi-choice answers and limited character box type answers both), flow chart type quizzes, and simple text essay submissions.
For browsing, and encouraging higher quality respones, the site could copy over and make use of the Stanford Encyclopedia of Philosophy (plus maybe an in-house one we all edit? wikipedia clone or wikipedia itself? ... again I am slightly basing this idea on iNaturalist and that's what they have for each taxon page). This way the people using the site can make common reference to definitions. This is the weakest part of the idea, as far as I'm concerned, because I hate the idea of making any source "our definitive source" or "our commonly accepted source" from the top down in the implementation.
And yet I don't want people running around defining shit stupid or for us all to get into a swamp of basically speaking incommensurable languages without admitting or being aware of it any more than is strictly necessary. I guess just having a lot of different references, and accepting people defining things differently (especially if they point it out clearly, ideally with a bit on why) could help split the difference there.
Responses should include:
- ticking of boxes in a big list of "best fit" labels and connections to existing philosophical traditions and "known thinkers", including sliders as options. Think someone fits entirely? Slide that to 100%. Think it's almost a match? Put like 90% and say why (or not) in a text response later on. Maybe there could be a small text box per one of these that says "Basically, why?" and gives them a short # of characters to say something right then, to stave off people forgetting later in the process or giving no response rather than a simple response.
- creation of quiz questions in a variety of formats (including the ones you outlined! I like the idea of flow-chart structured quizzes that narrow down what questions you get as you go, rather than comprehensive) to send back to the initial person, to refine their thought or to seek clarification, also, a chance to suggest reading and possibly request a response to some specific work or passage.
- essay responses!! These would be like someone writing about what they think you think, where they think it fits in existing philosophical traditions, or if it's any good, or all of the above. This is the rich feedback everyone who really wants to get into philosophical thought dreams of. Finally, you are the essay subject matter, and not just the essayist having to review something from Plato for the hundredth time.
There's something to make people stick around. The chance to hear - You did it!! You are a philosopher!!
And that is exactly the kind of encouraging text I think could be put up once people reach such and such numbers of responses rated above a certain level. I think it helps motivate most people to know there's some kind of check point built in everyone has agreed = basically, you participated well, if you like you can take your results thus far, and go.
This seems like a weird outlook to me, because I see it all as naturally endless and am comfortable just winging my own cut off point (if any), but how else will we make it less of an echochamber if not thinking of the preferences and needs of others quite different from ourselves? I don't want me and my 10,000 personality-clones around the world's philosophy, I want the world's philosophy represented. Of course, we can't get that, but we could give it a good try and its best chance.
So, about the rating system. Two main types, a subjective notion of quality without any respect to agreement (as much as that's possible) ... and we are in accord rating, which could be an interesting way to find people you agree with on some particular thing and see what else they think. This would also be like a % vote ranking but also people can leave brief feedback on quality or agreement if they so choose.
Having a feature like you discussed for categorical test making ... and a resulting "taxonomy of philosophy" both individually and collaboratively (open to edits, either just like all edits, or edits then moderated by the original test creator) could be nice. Really I think of things like this as providing a structure - tools and platform - for what the people then come and build (and rebuild).
