in general, trust the experts, but that it's reasonable to raise one's level of hesitation and wary skepticism when the field has some common negative indicators. Such as:
…The experts in the field routinely act in an epistemically sketchy manner, refusing to show work, share data or access to research objects or equipment, or reveal methodology. Perhaps they restrict access to or fully self-censor their results for reasons unrelated to inquiries into the truth of claims. Maybe they repeatedly apply nasty tactics, use the media or legal systems to attack heretic at the personal level, try to ruin their careers or lives, try to intimidate or bankrupt them with lawsuits, etc.
I bolded my favorite phrase. A shorter version of my original post is that I trust experts who show their work. Of course, if their work is based on dodgy models, that does not help much, but otherwise showing your work is the key for me. That is why the ability to communicate to a non-specialist is important.
Think of this in terms of the distinction between a prestige hierarchy and a dominance hierarchy. In chess, an ordinary player defers to the judgment of a grandmaster because the grandmaster has prestige. It is entirely voluntary on the part of the ordinary player. In a dominance hierarchy, the individual lower down defers to the person higher up in the hierarchy out of fear. Think of a criminal gang, where someone at the bottom rung is scared of the men above him.
Science is supposed to work like a prestige hierarchy. But sometimes we observe scientists making dominance moves, trying to intimidate or eliminate opposition. When experts in the field make dominance moves, that indicates to me that I should not trust them.
That is one reason that I do not like it when academics sign their names to documents that claim to represent expert consensus. "The consensus believes X” is a dominance move. The prestige move is to say “I believe X because of ____,” not to try to force something upon me as an alleged consensus.
Another issue that came up in the comments concerns whether to trust an expert on a topic that is outside of the expert’s field. I mean, do you trust a football player advertising insurance?
Actually, if an expert is really good at using Scout Mindset, I trust them outside of their area of expertise. Scott Alexander would be a classic example. Tyler Cowen would be another. In fact, I might trust Tyler more in an area that is outside of his expertise than in an area where he is really “dug in” on an issue.
"An expert is a man who has made all the mistakes which can be made, in a narrow field."
….Niels Bohr
One big thing I left out of that comment was the importance of "Skin in the Game". Or instead of "thinking in bets" you could say "Reporting In Bets", that is, verifiably or credibly putting a lot of their personal value at stake in the outcome of some prediction that can be objectively evaluated by outsiders in the short or medium term. An expert on nylon or polyester or efficient packaging theory may have some interesting ideas on the safest way to pack a parachute. But the opinions of the guy who sky-dives on a regular basis and who has to pack his own parachute are trustworthy in a qualitatively different way.