Getting csv string output from ODBC VBO

[NupurSood]

Hello All,

I have run into a challenge wherein I need to use "Data-ODBC" to fetch data from databricks table. The VBO only allows to get output as collection but since the dataset is large, I am running into system out of memory. It is not possible to build a logic to split data and bring it in so wanted to check if the below code can be altered to get a string output to have csv instead. 

OdbcDataAdapter adapter = new OdbcDataAdapter(queryString, conn);

adapter.SelectCommand.CommandTimeout = Convert.ToInt32(timeout);

DataSet dataSet = new DataSet();

try

{

adapter.Fill(dataSet);

status = true;

errMsg = "";

results = dataSet.Tables[0];

}

catch (Exception ex)

{

status = false;

errMsg = ex.Message;

results = null;

}

Would appreciate any guidance 

[John__Carter]

Rather than read in the full result set and then try to split it into pieces, which would still consume memory, another approach would be to extract smaller chunks using a series of queries. For example SELECT...WHERE ID BETWEEN 1 AND 1000, SELECT...WHERE ID BETWEEN 1001 AND 2000, etc.