(MONEY Magazine) – Disciples of socially responsible investing have long contended that you don't have to settle for second-rate returns if you put your money in a so-called ethical mutual fund to keep your conscience clear. But that view has just been challenged by a new analysis of the 10 funds with the longest track records. ''Someone who invested in the average-performing portfolio of a fund category would have earned 10% a year more than the ethical investor,'' says the study's author, Sam Mueller, associate professor of sociology at the University of Akron. What's more, the study also determined that the shares of ethical funds were more volatile than those of the competition. Mueller reviewed the performance of the funds for the five years through 1988 (except for Parnassus Fund, which had a four-year record). To make his sample as wide as possible, Mueller included five funds that might not meet many investors' definition of socially responsible investing -- namely, avoiding the securities of companies that invest in South Africa, pollute the environment or engage in unfair labor practices. Two money-market funds that Mueller reviewed -- Working Assets and Calvert Social Investment-Money Market -- differ from other money funds only slightly; for example, their managers don't buy Treasury issues, since they help finance defense spending. Three others are Pioneer stock funds that eschew only tobacco, gambling and alcohol companies. Still, even if these funds are excluded, the study's overall findings stand up: over a five-year period, socially conscious funds annually lag their competition by an average of one percentage point. True, two of the funds did beat their category averages: New Alternatives (516-466-0808), which invests in alternative energy and environmental companies, was up 82% over the five years; and Dreyfus Third Century (800-645-6561), a growth fund, returned 74%. But on a risk-adjusted basis, only Dreyfus Third Century outperformed its growth-stock category average. Its manager can buy defense stocks, though the fund hasn't held any for a year. Parnassus, Calvert Managed Growth and Pax World all underperformed their categories by more than a point. Supporters of ethical funds note that they are fairly new and that their managers' performance should improve over time. ''Socially responsible investing is just coming into its own,'' says Amy Domini, who plans to launch an index fund of 400 ethical stocks, the Domini Social Index Trust. Besides, for some investors seeking to make the world a better place, giving up 10% of their return may seem a small price to pay.