Yes, I'd say to talk his functional manager since he's the one managing the resources at the organization (and not the project level). It is important to note the following:
- The functional manager may refuse to do so (in this case, you are stuck with the resource unless you know someone higher in the food chain).
- The functional manager may not have an available resource that has the required skills to replace the old resource.
- The functional manager has an available resource, but his costs will be higher - which will increase the project's costs. In this case you will need an approval from the project sponsor to do this.
- The functional may suggest you try to resolve the productivity issue with the resource by yourself (e.g. "give the resource another chance!")