com.logicalclocks.hsfs

Class StorageConnector.BigqueryConnector

java.lang.Object

com.logicalclocks.hsfs.StorageConnector

com.logicalclocks.hsfs.StorageConnector.BigqueryConnector

Enclosing class:

StorageConnector

public static class StorageConnector.BigqueryConnector
extends StorageConnector

Nested Class Summary

Nested classes/interfaces inherited from class com.logicalclocks.hsfs.StorageConnector

StorageConnector.AdlsConnector, StorageConnector.BigqueryConnector, StorageConnector.GcsConnector, StorageConnector.HopsFsConnector, StorageConnector.JdbcConnector, StorageConnector.KafkaConnector, StorageConnector.RedshiftConnector, StorageConnector.S3Connector, StorageConnector.SnowflakeConnector

Field Summary

Fields inherited from class com.logicalclocks.hsfs.StorageConnector

storageConnectorApi, storageConnectorType

Constructor Summary

Constructors

Constructor and Description
BigqueryConnector()

Method Summary

Modifier and Type	Method and Description
List<Option>	getArguments()
String	getDataset()
String	getKeyPath()
String	getMaterializationDataset()
String	getParentProject()
String	getPath(String subPath)
String	getQueryProject()
String	getQueryTable()
Object	read(String query, String dataFormat, Map<String,String> options, String path) If Table options are set in the storage connector, set path to table.
void	setArguments(List<Option> arguments)
void	setDataset(String dataset)
void	setKeyPath(String keyPath)
void	setMaterializationDataset(String materializationDataset)
void	setParentProject(String parentProject)
void	setQueryProject(String queryProject)
void	setQueryTable(String queryTable)
Map<String,String>	sparkOptions() Set spark options specific to BigQuery.

Methods inherited from class com.logicalclocks.hsfs.StorageConnector

getDescription, getFeaturestoreId, getId, getName, getStorageConnectorType, refetch, setDescription, setFeaturestoreId, setId, setName, setStorageConnectorType, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

BigqueryConnector

public BigqueryConnector()

Method Detail

sparkOptions

public Map<String,String> sparkOptions()
                                throws IOException

Set spark options specific to BigQuery.

Specified by:

sparkOptions in class StorageConnector

Returns:

Map

Throws:

IOException - IOException

read

public Object read(String query,
                   String dataFormat,
                   Map<String,String> options,
                   String path)
            throws FeatureStoreException,
                   IOException

If Table options are set in the storage connector, set path to table. Else use the query argument to set as path.

Overrides:

read in class StorageConnector

Parameters:

query - query string

dataFormat - dataFormat

options - options

path - path

Returns:

Dataframe

Throws:

FeatureStoreException - FeatureStoreException

IOException - IOException

getPath

public String getPath(String subPath)

Specified by:

getPath in class StorageConnector

getKeyPath

public String getKeyPath()

setKeyPath

public void setKeyPath(String keyPath)

getParentProject

public String getParentProject()

setParentProject

public void setParentProject(String parentProject)

getQueryProject

public String getQueryProject()

setQueryProject

public void setQueryProject(String queryProject)

getDataset

public String getDataset()

setDataset

public void setDataset(String dataset)

getQueryTable

public String getQueryTable()

setQueryTable

public void setQueryTable(String queryTable)

getMaterializationDataset

public String getMaterializationDataset()

setMaterializationDataset

public void setMaterializationDataset(String materializationDataset)

getArguments

public List<Option> getArguments()

setArguments

public void setArguments(List<Option> arguments)