Senal

A similar concept : https://solar.lowtechmagazine.com/

No problem. Outside perspectives are usually interesting to explore.

I hate the idea that I’m falling for some sort of pseudoscience and weigh that against (a) how it tangibly helped me, and (b) whether we simply haven’t found the proper way to test its efficacy properly

Perhaps a different approach might help.

[ I will caveat the following with : i am not , in any way, qualified to give any psychological advice or medical suggestions, this is not that, it's just my personal opinion. ]

Rather than try and figure out if the test itself is flawed or not, look at the outcome instead.

Based on how you described it, it wasn't the specific methodology itself that was helpful to you.

You can take whatever positives you experienced and explore them completely independently.

Does it matter that you used a potentially flawed methodology to come to a useful conclusion about yourself ?

Would you mind elaborating on “control the context to eliminate bias and gaming” under this situation?

Sure, apologies if you already know any of this.

As with other scientific fields, there are guidelines and processes in place to evaluate the structure and approach for research.

iirc you don't technically have to adhere to them, but it will certainly be a point of industry and peer criticism if you don't, sometimes leading to papers not being accepted for journals and other more esoteric consequences.

This is one of the reasons proper peer review is important.

A basic example would be picking from (or narrowing to) an appropriate subset of the population.

If you were trying to perform research with the goal of evaluating the population as a whole, running your experiment exclusively with women between the ages of 18-25 would immediately be picked up as a reason the results can't be trusted (in terms of the stated goal).

A slightly less obvious example (for certain kinds of experiments) would be sentence structure and unconscious bias through contextual information.

When wording questions and examples it is easy to introduce a bias in the tone and word choice, which can affect the outcome of the research.

A real world example of the unconscious bias aspect is hiring discrimination : https://www.kcl.ac.uk/research/the-resume-bias-how-names-and-ethnicity-influence-employment-opportunities

A simplistic summary is that there is a bias (unconscious or otherwise) against people with "ethnic" sounding names on their resume.

There is, of course, more nuance to it than that, but still.

This is much less cut and dry because sometimes the bias is the thing being studied and forms a part of the test, which is why when creating these kinds of experiments the process is carefully evaluated and revised, hopefully by multiple people.

Another one you touched upon already is context, the time of day, life events, general disposition etc.

Good test design will try to account for as much of this as possible (though it's unlikely to remove it all entirely).

Obviously the more questions asked, the more granular the results can become, so I’ll grant that.

That's not always strictly true, quality is also important and there are diminishing returns on quantity, the length of a questionnaire can sometimes have it's own effect on the results for instance.

This relates to your final point: What would I consider to be the test’s objectives? For me, it’s an exercise in gleaning insight into one’s own personality; to help with reflection and introspection. To identify your strengths and weaknesses. In some sense, to provide some identity. I can’t tell you how I felt understood. I actually teared up while reading the analysis for the first time. As something of an outsider for much of my life it was like it filled in the missing pieces I long suspected and yet always doubted. Like I said I can’t speak for what others got out of the test, but it was the best therapy I ever received. (And for context, I read every other generalized group to make sure it wasn’t generalized astrological bullshit where every description could match every person, for which nothing came close).

It sounds like this experience was/is of great use to you. I've heard similar things about ADHD and ASD diagnoses.

Finding your tribe/place sounds great.

What i would say is that people who don't have this level of resonance with the results could well see it less favourably than you.

That isn't necessarily because they performed the test (or interpreted the result) incorrectly, it could just mean less to them.

I'm aware i'm cherry picking here.

Scientists do this all the time.

They do, with strict guidelines about how they can strictly control the context to eliminate bias and gaming (as much as they can anyway).

The only substantive arguments that I’ve seen made – and the only “debunking” aspects to this test revolve around veracity and validity – which is understandably concerning. But let’s unpack that: Do the results bear repeatability, and do what the results say reflect the reality of who that person is?

I could very well be reading this incorrectly but are you saying that veracity and validity are known concerns and then follow that up with "Can we verify? Are the results useful?"

I wouldn't consider restating the questions that represent the known concerns as unpacking said concerns.

misunderstanding of the test’s objectives.

Genuine question, what would you consider to be the test's objectives ?

This ^ hot take is brought to you by the same people who ask " If you don't believe in god what is stopping you from raping and murdering your whole neighborhood? "