Yes, you can make withdrawals. However, any funds you withdraw from your IFISA will lose the tax-efficient wrapper. Alternatively, you may prefer to transfer money to another ISA, in which case you will need to complete a transfer request with the new ISA provider.
Firstly, the Downing Bond(s) you hold need to pay interest or reach maturity so that you have a cash balance in your IFISA. Once you have a cash balance, you can then apply online to withdraw your money or contact your other ISA provider to request a transfer.