This sounds like a classic, introductory corporate finance question.
If interest payments on debt are tax deductible, increasing debt lowers corporate income taxes. Taking total firm cash flows as given, increasing debt effectively redirects cash flows from the government to equity holders. With no counteracting forces, firm owners will choose an all debt firm.
Why not do this? There are numerous objections you can raise to assumptions behind the above argument. For example:
- The cost of bankruptcy isn't zero.
- Excessive debt can cause agency problems (eg. underinvestment due to debt overhang) and reduce firm cash flows.
- There are numerous other ways to avoid taxes.
The list goes on... Anyway, these issues will be discussed in any corporate finance textbook or intro corporate finance class.
Some broader context...
The early academic, capital structure literature tried to explain optimal firm leverage as a tradeoff between tax benefits and bankruptcy costs. A lot though has happened since the early 1970s. There is now a tremendous quantity of corporate finance theory that posit alternative explanations for firm financing decisions.
Rather than give some simplistic survey of that theory here, I'd recommend the reader start with any introductory corporate finance text or class. The only thing else I'd mention is that the theory of firm capital structure is not a solved problem. Many existing theories are difficult to test, and to the extent that we can build empirical models to predict leverage ratios from theory, the $R^2$ you get aren't overwhelming.