I am interested in studying the effect of periodic boundary conditions on phonon frequencies from dynamical matrices. More specifically, my dynamical matrices are at the Gamma point only (one big supercell sytem).

I am sure there are several ways to do it but I am wondering if we could somehow turn off periodic boundary conditions in diagonalizing the dynamical matrix in GULP.

Would setting the domain size artificially large do the trick?

For instance, the normal supercell size is say 30 by 30 by 30 Angstroms and potential cutoff is 5 A.

To turn off the effect of periodic boundary condition, could I set the cell sizes as 100 by 100 by 100 while keeping the atomic positions the same as before?

It’s not quite clear what you’re trying to achieve here. You can turn off periodic boundary conditions very easily by just giving Cartesian coordinates and no cell input for GULP & you’ll have a finite cluster. From your description it seems like you are thinking more about minimum image rather than PBC perhaps? There is a minimum image keyword in GULP to do this (or make the cutoffs less than the cell length), though if you have electrostatics and use the Ewald sum then you’ll never exclude periodic effects. You should also note that the diagonalisation of the dynamical matrix is independent of PBC - the periodic information is in the dynamical matrix before it’s diagonalised.
Best regards
Julian