(CNN) -- The race-conscious admissions policy at the University of Texas appeared to be in trouble on Wednesday after the conservative Supreme Court majority repeatedly questioned its continued application and effectiveness.
The Justices heard oral arguments in an affirmative-action case that explores whether the flagship state university's admissions practices aimed at creating campus diversity violate the rights of some white applicants.
Abigail Noel Fisher sued the school after her application was rejected in 2008 when she was a high school senior in Sugar Land, Texas.
Fisher claimed the individualized, discretionary admission policies violated her rights, and favored African-American and Hispanic applicants over whites and Asian-Americans.
Fisher just missed the opportunity of automatic admission to the main campus at Austin for in-state students finishing in the top 10% of their high school graduating class. So, she had to compete in a separate pool. It is that selection process that is before the court.
The Supreme Court was clearly divided along ideological lines about whether affirmative action essentially has run its social and legal course, and should no longer be used in the way schools like Texas and others have done.
"You're trying to gut it," Justice Sonia Sotomayor said to Fisher's lawyer about the current legal precedent set in 2003.
She suggested that a university deserves some flexibility to create the kind of diverse campus environment it wants.
But Chief Justice John Roberts repeatedly asked, "What is the 'critical mass' of African-Americans and Hispanics at the university that you are working toward?"
When the school's lawyer said there was no specific number, Roberts pressed, "So how are we supposed to tell whether this plan is narrowly tailored to that goal? What is the logical end point?"
Justice Anthony Kennedy may prove the deciding or "swing" vote and could strike a compromise position -- possibly toss out the specific plan in question, while generally preserving the future use of affirmative action in more limited circumstances.
The school, with a 52,000-student body, defends its "holistic" policy of considering race as one of many factors -- including test scores, community service, leadership and work experience.
The Obama administration backed the school.
"Everyone competes against everyone else. Race is not a mechanical automatic factor. It's an holistic individualized consideration," said U.S. Solicitor General Donald Verrilli.
African-Americans in Texas as a whole represent about 12% of the population, but comprise about 5% of students at the University of Texas.
Much of the discussion during arguments surrounded the definition of "critical mass" -- how a university can use metrics to determine when it has reached the right mix of minority students to achieve diversity.
Justice Stephen Breyer wondered why his colleagues were even debating the issue following 2003 high court ruling that found state universities could narrowly set up admissions policies to consider an applicant's race.
The relevant high court case from nine years ago dealt with the University of Michigan's admission programs. The issue was divisive, with Justice Sandra Day O'Connor the swing vote upholding the law school's admissions criteria.
O'Connor, who is no longer on the bench, predicted affirmative action would no longer be needed in 25 years, offering a gentle push to institutions to keep that timeline in mind. Nearly a decade later, however, the newly-configured court may be poised to issue a more dramatic statement on the use of race.
The Texas school modified its admission policy to include race as one factor shortly after the 2003.
The court's more conservative members seized the initiative during arguments.
Roberts questioned how schools were to measure classroom diversity among students who come from mixed race families.
Justice Samuel Alito suggested it was unfair to place Asian-Americans into one group, when they come from a wide range of cultures-- Filipinos, Chinese, and Afghans among them.
"I thought that the whole purpose of affirmative action was to help students who come from underprivileged backgrounds, but you make a very different argument that I don't think I've ever seen before," he told Gregory Garre, representing the school.
Garre said the school wants minorities from different backgrounds.
"We go out of our way to recruit minorities from disadvantaged backgrounds," he said.
Kennedy jumped in.
"So what you're saying is that what counts is race above all," he said. "You want underprivileged of a certain race and privileged of a certain race. So that's race."
Kennedy added later: "I thought that the whole point is that sometimes race has to be a tie-breaker and you are saying that it isn't. Well, then, we should just go away. Then we should just say you can't use race, don't worry about it."
Fisher graduated this year from Louisiana State University.
A ruling is not likely before early next year.
The current case is Fisher v. University of Texas at Austin (11-345).