Consider having two backlogs
One of the tenants of agile development is transparency: for example, Scrum teams only work on items chosen by the product owner from the product backlog. If there are technical deficiencies in the code base it is up to the team to make the case to the customer or product owner. If the case is sufficiently unconvincing, then they don't get to do the work.
Is this really the best way to balance technical and business objectives?
In my experience working with teams, this approach has a couple of problems:
- Business stories generally provide value in the form of new revenue opportunities while technical stories typically provide value through reduced risk. Risk by nature is probabilistic, and teams are ill equipped to translate those risks into dollars.
- Product owners are not qualified to assess technical risk.
The consequences can be damaging to the organization:
- The team will find itself unable to stay current with its external dependencies. This is NOT the same as technical debt. Think of applications that still run on JDK 1.3 or MySQL 3.
- The business will find their team unable to deliver basic features due to architecture and platform issues.
- Sprint retrospectives become irrelevant. The information produced in those meetings is not actionable because the team can't justify the improvement projects to business.
The 3rd consequence is perhaps the worst in my opinion. Nothing kills a team's interest in Agile like giving them the opportunity for process improvement and then telling them they don't have time to implement the improvements they wish to make.
One potential solution is for the technology team to work from two backlogs: the standard product backlog owned by the product owner, and a second technical backlog owned by the senior technical manager (VP of Engineering, CTO, etc). The organization makes a decision about the blend of effort expended on the two queues. If your teams are estimating work in points this is fairly easy to manage.
This approach recognizes that the company needs to balance its investment in its existing software with its need to continue to provide enhancements to customers. It also gives senior technical management the ability to direct improvement efforts at a high level, accepting input from team members. The feedback gathered during retrospectives can generate items for the technical backlog.