C#: ListBox SelectedIndex changes itself when method returns

I have a problem with ListBox in c#(winforms). This ListBox contains the list of files(only pictures) in a specified directory like this:

private ListBox FileList = new ListBox();
private List<string> Directories = new List<string>
{
  @"some path",
  @"some other path",
  @"..."
};
private List<string> Pictures;

private int selectedDirectory = 0;
public int SelectedDirectory
{
  get { return selectedDirectory; }
  set
  {
    //the list of pictures is returned correctly
    Pictures = GetPictures(Directories[value]); 
    selectedDirectory = value;
    EventHandler handler = this.DirectoryChanged;
    // noone who subscribed to the event
    // changes FileList.SelectedItem or FileList.SelectedIndex
    if(handler != null)
      handler(this, EventArgs.Empty);
    this.SelectedPicture = Pictures.Count == 0 ? -1 : 0;
  }


this.DirectoryChanged += (sender, e) =>
{
  FileList.Items.Clear();
  FileList.Items.AddRange(Pictures);
};

private int selectedPicture
public int SelectedPicture
{
  get { return selectedPicture; }
  set
  {
    selectedPicture = value;
    if(value != -1)
      PictureBox1.Load(Pictures[value]);
    FileList.SelectedIndex = value;
  }
}

// after this method returns, FileList.SelectedIndex changes to a random value depending on what Directory was selected before the change
private void MainFormKeyDown(object sender, EventArgs e)
{
  if(e.KeyCode == Keys.NumPad5)
    SelectedDirectory --;
  if(e.KeyCode == Keys.NumPad2)
    SelectedDirectory ++;
} // the value of SelectedIndex changes after leaving this exact line

I tried to debug it and the result was that from setting the value of FileList.SelectedIndex in this.SelectedPicture it remained 0 until MainFormKeyDown returned.

What can cause this kind of behaviour and how can I solve it?

I have checked if I change the value anywhere else in the code or in any event subscription but not.

I also tried using ListView, but the result remained the same.

Now I'm out of ideas.